Being a programmer, rather that searching for some program on the Internet, I wrote my own quick and dirty command line tool for this task (mostly hard-coded paths etc.) in mid 2010. This year, I added GUI (where all the settings can be adjusted), some more date & time functions, and basically made the whole thing usable. And finally, beta release of PhotoMixer is available.

Download

Please report any bugs and problems you encounter. Ideas for new features as well!

Overview

PhotoMixer takes photo sources as input and mixes/sorts them. Photo source is a folder with photos that have different properties than some other source. Most often, it’s from a different person and/or taken with different camera. PhotoMixer takes all the photos from all sources and puts them all in one output folder with file names that contain timestamps (so that the files appear in the same order regardless if you sort them by name, type, or date).

The most important this is: it can adjust date & time of photos from each source automatically. As the result, you now have a folder with all the photos from your trip in the right order even though they originally came from ten people with all of them having wrong date & time setting in their cameras. Where’s the catch? Well, there should be one photo that each of these people took at nearly the same time. These reference photos are then used by PhotoMixer to synchronize the sources.

Each source can be synchronized using reference photo or just by explicit time shift. Photo date and time is taken from embedded EXIF metadata that’s generated by digital camera and inserted in JPEG file. Only when, for some reason, EXIF is not present, file system date is used instead. When using reference photo synchronization one of the sources must be designated as main reference source. That is the source whose dates will not be modified and all other sources will be shifted in regard to date differences between their reference photo and that of the main source.

PhotoMixer also offers some useful tools for related tasks.

Usage Guide

PhotoMixer employs tabbed user interface where the first tab contains output settings and log. Additional tabs define photo sources. When you start the tool for the first time new source is created, and you can start setting it up and adding more sources. When you are done with sources, go to Settings tab, adjust output options, and finally click Run button.

Photo source options

For each source you can select folder with photos and type of synchronization. For sync by reference photo you must select this photo and for explicit shift you provide positive or negative time in seconds.

Settings and output tab

On Settings tab select the output folder, base name of the generated files (will be named “BaseName TimeStamp.jpg”), and main reference source used for synchronization.

All the options and source definitions can be saved for later usage and restored using the Save and Load actions in the Tools menu. There is also tool for clearing the output folder and two little more complex tools: Set Photo Date & Time and Set File Time from Photo Metadata.

"Set Photo Date & Time" tool

If you totally lost all date & time info about some photos and you want at least something (e.g. you use some photo organizer and want some filtering based on year & month to work) you can generate it using Set Photo Date & Time tool. Some base date & time is selected and each photo can have optional time increment (e.g. first photo has “base time”, second has “base time + 5 minutes”, another one has “base time + 10 minutes”, etc.).

Set File Time from Photo Metadata tool can be used to restore file system date & time according to metadata inside the photo.

Nice to have features for future version

Some are definitely going to be included, some are more like pipe dreams.

• Detect blurred photos that you just want to delete.
• Detect similar images – often more people take a photo of the same thing but you want to keep just one (the best looking one).
• Resampling of output photos to different resolution. Although this is a lossy operation, it has its uses.
• Create embedded JPEG thumbnails where missing (they’re embedded by cameras but some photo editing program could remove it).
• Lossless rotation of MJPEG videos from cameras (90/180/270 rotations).
• Strip junk data streams (e.g. after lossless rotation in Irfan the old data stream is kept in the file resulting in 2x bigger file).
• Linux and Mac OS X version (at least with CLI).
• Something to deal with camera orientation (when taking the photo). Some cameras have positioning sensor and store this information in metadata so the program could rotate them accordingly. For others, some GUI that allows simple and fast manual rotations would be nice. Probably redundant: probably just do it in Live Gallery/Picassa/whatever.

The End

Sorting of about 3000 photos too about 2 minutes on my desktop (nearly all of that time is spent reading EXIF metadata from the files). Here are some photos from the “last batch” (panoramas made in Microsoft ICE):

Half floats first appeared in early 2000s as samples in images and textures. Floats provide higher dynamic range than what is available with regular 8bit or 16bit integer samples. On the other hand, commonly used single and double precision floats have much higher memory cost per pixel. Half floats have more reasonable memory requirements and their precision is adequate for many usages in imaging.

16bit float formats have been supported by ATI and NVidia GPUs for many years. I’m not sure about other IHVs but at least Direct3D 10 capable GPUs should all support it.

Read on if your interested how to convert between half and single precision floats (Object Pascal code).

Half/Single Conversion code

Finally, here’s the code for converting to half floats and back to single precision float. It’s based on C++ code from OpenEXR library (half class).

First some types and constants

Note that THalfFloat type is just an alias for Word.

type
  THalfFloat = type Word;

const
  HalfMin:     Single = 5.96046448e-08; // Smallest positive half
  HalfMinNorm: Single = 6.10351562e-05; // Smallest positive normalized half
  HalfMax:     Single = 65504.0;        // Largest positive half
  // Smallest positive e for which half (1.0 + e) != half (1.0)
  HalfEpsilon: Single = 0.00097656;
  HalfNaN:     THalfFloat = 65535;
  HalfPosInf:  THalfFloat = 31744;
  HalfNegInf:  THalfFloat = 64512;

Single precision float to half

function FloatToHalf(Float: Single): THalfFloat;
var
  Src: LongWord;
  Sign, Exp, Mantissa: LongInt;
begin
  Src := PLongWord(@Float)^;
  // Extract sign, exponent, and mantissa from Single number
  Sign := Src shr 31;
  Exp := LongInt((Src and $7F800000) shr 23) - 127 + 15;
  Mantissa := Src and $007FFFFF;

  if (Exp > 0) and (Exp < 30) then
  begin
    // Simple case - round the significand and combine it with the sign and exponent
    Result := (Sign shl 15) or (Exp shl 10) or ((Mantissa + $00001000) shr 13);
  end
  else if Src = 0 then
  begin
    // Input float is zero - return zero
    Result := 0;
  end
  else
  begin
    // Difficult case - lengthy conversion
    if Exp <= 0 then
    begin
      if Exp < -10 then
      begin
        // Input float's value is less than HalfMin, return zero
         Result := 0;
      end
      else
      begin
        // Float is a normalized Single whose magnitude is less than HalfNormMin.
        // We convert it to denormalized half.
        Mantissa := (Mantissa or $00800000) shr (1 - Exp);
        // Round to nearest
        if (Mantissa and $00001000) > 0 then
          Mantissa := Mantissa + $00002000;
        // Assemble Sign and Mantissa (Exp is zero to get denormalized number)
        Result := (Sign shl 15) or (Mantissa shr 13);
      end;
    end
    else if Exp = 255 - 127 + 15 then
    begin
      if Mantissa = 0 then
      begin
        // Input float is infinity, create infinity half with original sign
        Result := (Sign shl 15) or $7C00;
      end
      else
      begin
        // Input float is NaN, create half NaN with original sign and mantissa
        Result := (Sign shl 15) or $7C00 or (Mantissa shr 13);
      end;
    end
    else
    begin
      // Exp is > 0 so input float is normalized Single

      // Round to nearest
      if (Mantissa and $00001000) > 0 then
      begin
        Mantissa := Mantissa + $00002000;
        if (Mantissa and $00800000) > 0 then
        begin
          Mantissa := 0;
          Exp := Exp + 1;
        end;
      end;

      if Exp > 30 then
      begin
        // Exponent overflow - return infinity half
        Result := (Sign shl 15) or $7C00;
      end
      else
        // Assemble normalized half
        Result := (Sign shl 15) or (Exp shl 10) or (Mantissa shr 13);
    end;
  end;
end;

Half to single precision float

function HalfToFloat(Half: THalfFloat): Single;
var
  Dst, Sign, Mantissa: LongWord;
  Exp: LongInt;
begin
  // Extract sign, exponent, and mantissa from half number
  Sign := Half shr 15;
  Exp := (Half and $7C00) shr 10;
  Mantissa := Half and 1023;

  if (Exp > 0) and (Exp < 31) then
  begin
    // Common normalized number
    Exp := Exp + (127 - 15);
    Mantissa := Mantissa shl 13;
    Dst := (Sign shl 31) or (LongWord(Exp) shl 23) or Mantissa;
    // Result := Power(-1, Sign) * Power(2, Exp - 15) * (1 + Mantissa / 1024);
  end
  else if (Exp = 0) and (Mantissa = 0) then
  begin
    // Zero - preserve sign
    Dst := Sign shl 31;
  end
  else if (Exp = 0) and (Mantissa <> 0) then
  begin
    // Denormalized number - renormalize it
    while (Mantissa and $00000400) = 0 do
    begin
      Mantissa := Mantissa shl 1;
      Dec(Exp);
    end;
    Inc(Exp);
    Mantissa := Mantissa and not $00000400;
    // Now assemble normalized number
    Exp := Exp + (127 - 15);
    Mantissa := Mantissa shl 13;
    Dst := (Sign shl 31) or (LongWord(Exp) shl 23) or Mantissa;
    // Result := Power(-1, Sign) * Power(2, -14) * (Mantissa / 1024);
  end
  else if (Exp = 31) and (Mantissa = 0) then
  begin
    // +/- infinity
    Dst := (Sign shl 31) or $7F800000;
  end
  else //if (Exp = 31) and (Mantisa <> 0) then
  begin
    // Not a number - preserve sign and mantissa
    Dst := (Sign shl 31) or $7F800000 or (Mantissa shl 13);
  end;

  // Reinterpret LongWord as Single
  Result := PSingle(@Dst)^;
end;

Deskewing some smart paper

My approach is fairly common for this problem – rotation angle is first determined using Hough transform and then the image is rotated accordingly. Classical Hough transform is able identify lines in the image and it was later extended to allow detection of any arbitrary shapes.

Lines of text can be thought of as horizontal lines in the image. In a skewed scanned document all the lines will be rotated by some small angle. We can start with the equation of the line y = k · x + q. Since we’re interested in the angle, we can rewrite it as y = (sin(α) / cos(α)) · x + q. Finally, we can rearrange it as y · cos(α) − x · sin(α) = d. Now every point [x, y] in the image can have infinite number of lines going through it, where each is defined by two parameters: angle α and distance from the origin d.

We want to consider lines only for certain points of input image. Ideally, that would be the base lines on which the “text is sitting”. Simple way of determining these points is to check for black pixels which have white pixels just below them. Now for each of the classified points, we determine parameters α and d for all the lines that go through them. To get some finite number of lines, we calculate d for angles α from a certain range (I use angle step of 0.1 degrees). We want to find a line that intersects as many classified points as possible – an accumulator is used to store “votes” for each calculated line. For each point that is believed to be on the text base line, we add one vote for each line that intersects it. At the end, we find the top lines that have the most votes. Ideally, these are the base lines of all lines of text in the document. Finally, we get the rotation angle by averaging angle α of the top lines and rotate the whole image accordingly.

Important part is that one: “check for black pixels which have white pixels just below”. What’s black and white is determined by comparing value of the current pixel against some given threshold. For images where background is plain white and the text is black it’s easy just to use 0.5 as the threshold. But when the background/foreground distinction is not so sharp calculating the threshold adaptively based on the current image can be very useful. Deskew supports both adaptive threshold calculation as well as specifying constant threshold as command line parameter.

Deskewing some math exercise

Implementation is written in Object Pascal and uses Imaging library for reading and writing various image file formats. There are precompiled binaries for 32bit Windows and Linux, other platforms can be built from sources using Free Pascal compiler (you need Imaging library for compilation). Archive also contains few test images.

One way is to override DoDraw method of TImageList and change the code that draws disabled images. You can do regular RGB to grayscale conversion here or let  Windows draw it for you in grayscale with nearly no work on your part. You can do this by using ImageList_DrawIndirect with ILS_SATURATE parameter. Note that this works only on Windows XP and newer and for 32bit images only. For older targets or color depths doing your own RGB->grayscale conversion is an option (good idea would probably be to cache converted grayscale images somewhere so they won’t need to be converted on every draw call).

Here’s the code of DoDraw method using  ILS_SATURATE:

type
// Descendant of regular TImageList
TSIImageList = class(TImageList)
protected
  procedure DoDraw(Index: Integer; Canvas: TCanvas; X, Y: Integer;
    Style: Cardinal; Enabled: Boolean = True); override;
end;

procedure TSIImageList.DoDraw(Index: Integer; Canvas: TCanvas; X, Y: Integer;
  Style: Cardinal; Enabled: Boolean);
var
  Options: TImageListDrawParams;

  function GetRGBColor(Value: TColor): Cardinal;
  begin
    Result := ColorToRGB(Value);
    case Result of
      clNone: Result := CLR_NONE;
      clDefault: Result := CLR_DEFAULT;
    end;
  end;

begin
  if Enabled or (ColorDepth <> cd32Bit) then
    inherited
  else if HandleAllocated then
  begin
    FillChar(Options, SizeOf(Options), 0);
    Options.cbSize := SizeOf(Options);
    Options.himl := Self.Handle;
    Options.i := Index;
    Options.hdcDst := Canvas.Handle;
    Options.x := X;
    Options.y := Y;
    Options.cx := 0;
    Options.cy := 0;
    Options.xBitmap := 0;
    Options.yBitmap := 0;
    Options.rgbBk := GetRGBColor(BkColor);
    Options.rgbFg := GetRGBColor(BlendColor);
    Options.fStyle := Style;
    Options.fState := ILS_SATURATE; // Grayscale for 32bit images

    ImageList_DrawIndirect(@Options);
  end;
end;

Important note: For ILS_SATURATE to work correctly source image files must be 32bit with proper alpha channel data, setting color depth of TImageList to 32bit is not enough! If you don’t see any images drawn this is probably the cause: 8/24bit image is loaded from file and then inserted into 32bit TImageList. As there is no alpha channel data in source image it is drawn as fully transparent so you don’t see anything.

It does in fact. SHR treats its first operand as unsigned value even though it is a variable of signed type whereas >> takes the sign bit into account. In the function I converted the operand for right shift was often negative and Delphi’s SHR just ignored the value of the sign bit.

A Bit of Code

int a, b1, b2;
a = -512;
b1 = a >> 1;
b2 = a / 2;

After running this C code both b1 and b2 have a value of -256.

var A, B1, B2: Integer;
A := -512;
B1 := A shr 1;
B2 := A div 2;

This Delphi code however yields different result: B2 is -256 as expected but B1 has a value of 2147483392.

A Bit of Assembler

Assembler output of C code:

Unit1.cpp.22: b1 = a >> 1;
mov eax,[ebp-$0c]
sar eax,1
mov [ebp-$10],eax
Unit1.cpp.23: b2 = a / 2;
mov edx,[ebp-$0c]
sar edx,1
jns $00401bb9
adc edx,$00
mov [ebp-$14],edx

Assembler output of Delphi code:

Unit1.pas.371: B1 := A shr 1;
mov eax,[ebp-$0c]
shr eax,1
mov [ebp-$1c],eax
Unit1.pas.373: B2 := A div 2;
mov eax,[ebp-$0c]
sar eax,1
jns $00565315
adc eax,$00
mov [ebp-$20],eax

As you can see, asm output of C and Delphi divisions is identical. What differs is asm for shift right operator. Delphi uses shr instruction whereas C uses sar instruction. The difference: shr does logical shift and sar does arithmetic one.

The SHR instruction clears the most significant bit (see Figure 6-7 in the Intel Architecture Software Developer’s Manual, Volume 1); the SAR instruction sets or clears the most significant bit to correspond to the sign (most significant bit) of the original value in the destination operand.

Quoted from: http://faydoc.tripod.com/cpu/shr.htm

First difference you may notice (compared to Pascal) is case sensitiveness of Modula-2, for instance all reserved words are upper cased. You don’t have to write so many begin-end statements though. Another C-like trait is importance of header files (definition modules) order during compiling – just change it a little bit in a large project and whole thing breaks down.

More info about Modula-2 if you’re interested: http://www.modula2.org/.