Deskewing some smart paper

My approach is fairly common for this problem – rotation angle is first determined using Hough transform and then the image is rotated accordingly. Classical Hough transform is able identify lines in the image and it was later extended to allow detection of any arbitrary shapes.

Lines of text can be thought of as horizontal lines in the image. In a skewed scanned document all the lines will be rotated by some small angle. We can start with the equation of the line y = k · x + q. Since we’re interested in the angle, we can rewrite it as y = (sin(α) / cos(α)) · x + q. Finally, we can rearrange it as y · cos(α) − x · sin(α) = d. Now every point [x, y] in the image can have infinite number of lines going through it, where each is defined by two parameters: angle α and distance from the origin d.

We want to consider lines only for certain points of input image. Ideally, that would be the base lines on which the “text is sitting”. Simple way of determining these points is to check for black pixels which have white pixels just below them. Now for each of the classified points, we determine parameters α and d for all the lines that go through them. To get some finite number of lines, we calculate d for angles α from a certain range (I use angle step of 0.1 degrees). We want to find a line that intersects as many classified points as possible – an accumulator is used to store “votes” for each calculated line. For each point that is believed to be on the text base line, we add one vote for each line that intersects it. At the end, we find the top lines that have the most votes. Ideally, these are the base lines of all lines of text in the document. Finally, we get the rotation angle by averaging angle α of the top lines and rotate the whole image accordingly.

Important part is that one: “check for black pixels which have white pixels just below”. What’s black and white is determined by comparing value of the current pixel against some given threshold. For images where background is plain white and the text is black it’s easy just to use 0.5 as the threshold. But when the background/foreground distinction is not so sharp calculating the threshold adaptively based on the current image can be very useful. Deskew supports both adaptive threshold calculation as well as specifying constant threshold as command line parameter.

Deskewing some math exercise

Implementation is written in Object Pascal and uses Imaging library for reading and writing various image file formats. There are precompiled binaries for 32bit Windows and Linux, other platforms can be built from sources using Free Pascal compiler (you need Imaging library for compilation). Archive also contains few test images.

Few days ago I tried C++ Builder 2010 and was pleasantly surprised. It worked! I tried just the library core for now (ImagingTypes, Imaging, ImagingFormats, Pascal only file handlers, etc.) and it works without problems. I’m not sure which C++ Builder version is required for successful compile though. Versions 6 and 2006 stopped with internal error, 2010 worked, and there are 2 other versions between.

Anyway, I’ll try to check out most of the library until the 2010 trial expires for me. Hmm, I’m wondering how many people use C++ Builder for C++ development – I’ve never did something serious in it, basically just to get object files usable by Delphi – so I have no idea if it’s ok.

PS: Another C++ Builder related news – patch for OpenJpeg library to get it compiling in BCB is posted here.

So what can you expect in the next version of PasJpeg2000 library?
Higher GCC optimization levels should make it a lot faster when using Free Pascal (particularly in Mac OS X where O0 was used). Irreversible DWT transformation produces smaller lossy files than current PasJpeg2000 version and with optional MCT (multicomponent transform – basically RGB>YCbCr) you get even smaller ones. There’s now also a patch that enables OpenJpeg to get palettes from JP2 files so indexed JPEG 2000 images could be supported too. And finally, there are some bug fixes (wrong reconstruction of subsampled files, …).

More info and downloads at Imaging’s homepage.

Mainly for testing purposes during implementation of these new methods, I want to create extension for Imaging that compares two images (original and one reconstructed from compressed original) and measures PSNR and some other quality metrics.

I’m also thinking of implementing DXT5 based format using YCoCg colorspace and PVRTC (texture compression currently used in iPhone).

Few links if you’re interested:

• Brief DXTC overview: http://en.wikipedia.org/wiki/S3_Texture_Compression
• More technical details: http://www.opengl.org/registry/specs/EXT/texture_compression_s3tc.txt
• YCoCg-DXT related: http://developer.nvidia.com/object/real-time-ycocg-dxt-compression.html

Here’s some quick comparison of DXT compressors (click the image to see full size). Value in brackets is MSE (mean square error) – lesser number means compressed image is more similar to original.

Now I’m gonna extend PNG saver to allow saving multi images to simple APNG files, much like MNG saving works now. There’s really not a good unified way how to pass some more information to image file savers in current library design – that’s one of TODOs for new Imaging architecture.

There is already PNG-like chunk based format for animations called MNG (already supported by Imaging – at least the basic features). However, MNG is quite complex format and its support among browsers and image viewers/editors is lacking. Code library supporting all MNG features is huge.

APNG on the other hand is just an extension of PNG and its implementation is not so complex. I’m going to load only the raw frames from files at first and see what will have to be done to support animating the frames next. Canvas class will have to be used here for alpha blending subsequent frames to previous ones. I’ll add option to turn the animating on/off just like it is available for animated GIF files.

More info about APNG: http://www.animatedpng.com and https://wiki.mozilla.org/APNG_Specification.

After not so straightforward installation of Mac OS X in VMWare I fixed the issue just by changing number “24” to “32” in the code (TRawImage.Description.Depth field, LCL raw image to TBitmap conversion). Apparently, Carbon created bitmap with 6 bits per color channel. Now I just need to check if 24->32 change doesn’t break anything when using other LCL widget sets (I’m sure there was a reason for 24bits since I vaguely remember 32bits were there few years ago) – so maybe conditional compilation will be needed here.

Another issue I noticed is that LCL Imager demo couldn’t load default image (Tigers.jpg) that is displayed when it is started without parameters. Demo uses relative path to the image but (from Demos/Bin to Demos/Data directory). Mac OS X application LCLImager.app is placed in Demos/Bin directory by Lazarus but it is not a simple single file. It’s a directory itself and actual demo executable is located somewhere inside. I’ve not really decided on solution yet. Maybe embed the image in the executable as resource?

See the difference?

I decided to remove Kylix support (CLX graphic classes, project files, build scripts, core library still compiles). It’s not working properly on many (all?) current Linux distros (so I can’t test) and it was abandoned by Borland/Codegear quite some time ago. It was nice to have DCC compiler in Linux and it also made Borland to make Delphi RTL crossplatform. There are rumors about crosscompiling features in upcoming Delphi releases (in 2010?)  so maybe we’ll see DCC in Linux again.

Instead of Kylix project files there are new ones for Delphi 2009. Imaging itself didn’t require many fixes to compile and work with Delphi 2009,  most of them were related to text-based file format loaders (XMP, PNM) and external libraries (JpegLib, ZLib).

More info and downloads at Imaging’s homepage.

I’m planning revamping this whole site. Maybe creating some sort of simple CMS myself or just some better way to handle the projects. I don’t enjoy coding in PHP but there’s not much choice.

• New image file format support: GIF (native), TIFF (libtiff), PSD (native).
• New image data format: BTC (block truncation coding).
• File format compatibility fixes (for BMP, JPEG, and DDS)
  and platform/compiler support fixes (Win64, UNIX, FPC, …).
• Bug fixes and some enhancements (buffered file IO, …).

Visit Vampyre Imaging Library Homepage for more news and downloads or Imaging Forum.

Vampyre Imaging Library Homepage with news and downloads.
Imaging Forum with more news.

File formats in extras will also include JPEG2000. Reason why it won’t be in the core library is this: there is no native Pascal library for loading/saving JPEG2000 images (at least I haven’t found one) so Imaging will link to C library compiled to object files (OpenJPEG most probably). Therefore it won’t work on all platforms supported by Imaging (and thus cannot be included in core library).

There is also PCX loader, there will be PNM (for PBM, PGM, PPM, and PAM) loader/saver, and maybe even GIF support (if there will be enough time to implement it). I have converted my old Daggerfall image/texture loaders to Imaging too and added Redguard and BattleSpire support.

First version of extras package will be released around the end of 2006. New Imaging release will be made too because stuff in extras depends on some changes made in core library since 0.20 release. Current working version of Imaging and extras package is in project’s SubVersion repository.

Visit Vampyre Imaging Library homepage

Main news/changes in 0.20.0:

• New TImagingCanvas class with methods for drawing objects and applying effects was added (although there is not much functionality yet).
• New image data formats based on 16bit half-float (FP16) type has been added and OpenGL and Direct3D extensions can now create textures in these formats too.
• Imaging now compiles and works on 64bit CPUs (Linux on AMD64).
• High level interface classes were extended and updated as well as VCL/CLX/LCL support and low level interface.
• Some old demos were removed, new added, and rest was updated.

You can learn more about Imaging at its homepage.