Deskew CLI Tool v1.30 Released

New version of Deskew command line tool is ready. You can find general info about Deskew here Deskew Tools or check out README.

Main improvement for this version is better quality of rotated images. They're less blurry with default filtering, especially when rotated by less than one degree. You can now also select other filters, the choices are: nearest (no filtering, very fast), bilinear - default, bicubic, and Lanczos (overal best quality, pretty slow).
Deskew Rotation Quality Comparison

Change list for v1.30

  • fix #15: Better image quality after rotation - better default and also selectable nearest|linear|cubic|lanczos filtering
  • fix #5: Detect skew angle only (no rotation done) - optionally only skew detection
  • fix #17: Optional auto-crop after rotation
  • fix #3: Command line option to set output compression - now for TIFF and JPEG
  • fix #12: Bad behavior when an output is given and no deskewing is needed
  • libtiff in macOS is now picked up also when binaries are put directly in the directory with deskew
  • text output is flushed after every write (Linux/Unix): it used to be flushed only when writing to device but not file/pipe.

Downloads

  Deskew v1.30
» 4.3 MiB - 20,680 hits - June 19, 2019
Command line tool for deskewing scanned documents. Binaries for several platforms, test images, and Object Pascal source code included.

GitHub Release

GUI Frontend for Deskew

I've created a simple GUI frontend for Deskew. Now it's easier to process many files without writing shell scripts. It needs the command line tool which is called for the each input file. You can set the basic and most of the advanced options for deskewing in the GUI.

Prebuilt executables for Windows and Linux are available in the download - you just place them to the same folder as the command line tool. Version for macOS is a bit more convenient - it's a self-contained app bundle with CLI tool already inside and all placed in DMG image. You can also set the explicit path to the command line tool in the program itself.

The GUI is written in Lazarus so it may not be a best native-looking application out there but it saved me some time - there wouldn't be any GUI if it would be a big time sink.

Download

  DeskewGui v0.90
» 4.1 MiB - 6,304 hits - March 18, 2019
GUI frontend for Deskew command line tool. Prebuilt binaries for Windows, macOS, and Linux. Windows and Linux versions need Deskew command line tool binaries.

Remember that for Windows and Linux you also need Deskew command line tool if you don't have it already:

  Deskew v1.30
» 4.3 MiB - 20,680 hits - June 19, 2019
Command line tool for deskewing scanned documents. Binaries for several platforms, test images, and Object Pascal source code included.

Screenshots

Basic options and files to deskew in Deskew GUI in Windows

Advanced options in Deskew GUI in macOS

Deskewing in progress in Deskew GUI in Windows

Output of the command line tool in Deskew GUI in Linux

Bug Reports And Source Code

GUI is in the same repository as the command line tool, you can find the links here Deskew Tools.

Debugging with CDB in QtCreator Agonizingly Slow

A few years ago I wrote about my Problems with CDB Debugger in QtCreator. And a few months ago debugging started getting slower and slower and it finally reached a point where the program would need a minute or more just to start and stopping for a breakpoint was almost a coffee break situation. The old remedy wasn't helping anymore — this was something new.

While searching for a cure I stumbled upon Qt mailing list thread where the culprit was identified. It's Microsoft's symbol server!

Just go to Options -> Debugger -> CDB Paths tab and make sure srv* path is below cache* (or just remove it altogether). Looks like server symbols are preferred over local symbol cache if on top and that server is now terribly slow.

Deskew Tool v1.25 Released

New version of Deskew command line tool is ready. You can find general info about Deskew here Deskew Tools.

Change List for Deskew 1.25

  • fixed issue #6: Preserve DPI measurement system (TIFF)
  • fixed issue #4: Output image not saved in requested format (when deskewing is skipped)
  • dynamic loading of libtiff library - adds TIFF support in macOS when libtiff is installed
  • fixed issue #8: Cannot compile in Free Pascal 3.0+ (Windows) - Fails to link precompiled LibTiff library
  • fixed issue #7: Windows FPC build fails with Access violation exception when loading certain TIFFs (especially those saved by Windows Photo Viewer etc.)
  • Linux ARM build is now also included in the release

Download

  Deskew v1.30
» 4.3 MiB - 20,680 hits - June 19, 2019
Command line tool for deskewing scanned documents. Binaries for several platforms, test images, and Object Pascal source code included.

Deskew Tool v1.20 Released

New version of Deskew command line tool is ready. You can find general info about Deskew here Deskew Tools.

Change List for Deskew 1.20

  • much faster rotation, especially when background color is set (>2x faster, 2x less memory)
  • can skip deskewing step if detected skew angle is lower than parameter (possible speedup when processing large batches)
  • new option for timing of individual steps
  • fix: crash when last row of page is classified as text
  • misc: default back color is now opaque black, new forced output format "rgb24",
    background color can define also alpha channel, nicer formatting of text output

Download

  Deskew v1.30
» 4.3 MiB - 20,680 hits - June 19, 2019
Command line tool for deskewing scanned documents. Binaries for several platforms, test images, and Object Pascal source code included.

GEOTRANS in .NET and Java

GEOTRANS

GeoTrans (Geographic Translator) is a software that allows you to convert geographic coordinates among a variety of coordinate systems, map projections, and datums. It is made available by U.S. National Geospatial Intelligence Agency at no cost and free of copyright restrictions.

It is written in C (recent versions are in C++) and I've seen it used in software inside various surveying instruments. I worked on a supporting software for these devices and needed to do conversions of measured coordinates etc. So naturally, I wanted to use the tried and tested GeoTrans in my software as well. Only I needed to use it in .NET and Java…

Continue reading

Problems with CDB Debugger in QtCreator

Some time ago Locals and Expressions view in QtCreator just stopped working for me. No locals were listed when the program stopped on a breakpoint, watches did not show the values, hovering with the mouse over a variable in the editor did not show its value in a tooltip. No fiddling with IDE options helped. This was on Windows using compiler from Visual C++ and CDB as a debugger.

It was quite annoying but I was mostly working on UI at that time and could live with occasional dumping of few variables into a log. Few weeks later I moved to some math heavy stuff and the situation became desperate — I had to fix this!

At first, I suspected the debugger or the compiler. Since reinstalling and trying different versions did not help I also tried different versions of Qt and QtCreator itself. Still broken. And Googling for broken CDB only revealed problems with CDB getting unusably slow.

After QtCreator reinstall, I noticed the settings were preserved from the old version. So maybe they are somehow corrupted?  I found them in C:\Users\username\AppData\Roaming\QtProject\qtcreator\, made a backup of that folder, and deleted it. And it started working! Locals and Expressions were back!

Of course, I also wanted my old settings back. So I switched back to the old settings and started QtCreator again and to my surprise it still worked! I took a closer look and saw that the problem lies in the session file (*.qws) — when I previously deleted the settings folder QtCreator created a new session for me and this one I reopened in the IDE after the switch back to the old settings (just copied the files into the folder so the new session file stayed there).

So I opened the session file with an intention to start deleting suspicious stuff until it works. It's a XML file and apart from one big base64 blob there is not much else. Besides a breakpoint list, the only thing that seemed related to debugging was a list of maybe ten old watch expressions (which were not listed in the IDE anymore!). I deleted it and voilà, I could see the values of locals and variables again!

After a few weeks, debugging started to get slow. Stepping over code took a few seconds and values of the variables showed up only after considerable time (tens of seconds). This time I knew what to do, deleting watch expressions from the session file helped again. So I came to the conclusion that in the first case the values could eventually show up after a few minutes (stepping over code was not slowed down though). It was just very very slow. Then the Googled complaints about slow CDB made more sense to me. And indeed, I found out someone fixed it the same way but I was not paying attention to just slow debugging before, I was looking for broken debugging!

Recapitulation

  1. Debugging in QtCreator using CDB in Windows is very slow and/or values of locals and expressions never show up.
  2. Go to the folder where QtCreator stores your session file (C:\Users\username\AppData\Roaming\QtProject\qtcreator\).
  3. Open the session file (*.qws) in text editor and look for this XML subtree:
    <data>
        <variable>value-Watchers</variable>
        <valuelist type="QVariantList">
            <value type="QString">some expression 1</value>
            <value type="QString">some expression 2</value>
            <value type="QString">some expression 3</value>
        </valuelist>
    </data>
    
  4. Delete this subtree (or just the expressions) and save the file.
  5. Start QtCreator, open the fixed session, and do some debugging.

Edit: If this doesn't help check also Debugging with CDB in QtCreator Agonizingly Slow.

.NET and Java: Generating Interoperable AES Key and IV

Let's assume we want to generate encryption key and initialization vector (IV) for AES encryption based on some passphrase. And we want to be able to generate the same key and IV for the same passphrase in .NET and Java - maybe we have Android app written in Java that needs to decrypt message from ASP.NET web app.

In .NET, Rfc2898DeriveBytes class is often used to derive keys of specified length according to given passphrase, salt, and iteration count (RFC2898 / PBKDF2). For 256-bit key and 128-bit key it is as simple as this:

var keyGen = new Rfc2898DeriveBytes(passwordBytes, 
    saltBytes, iterationCount);

byte[] key = keyGen.GetBytes(256 / 8);
byte[] iv = keyGen.GetBytes(128 / 8);

Fortunately, PBKDF2 implementation is also built-in in Java:

SecretKeyFactory factory = 
    SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
KeySpec spec = new PBEKeySpec(passwordChars, 
    saltBytes, iterationCount, 256);
SecretKey secretKey = factory.generateSecret(spec);
byte[] key = secretKey.getEncoded();

We have the same key byte array, albeit with some more typing. And how about the initialization vector now? One could think that creating new PBEKeySpec with a length of 128 is the way to go. I know I did.

However, you would just get the same bytes as for the key (the first half of them). This key derivation algorithm is deterministic so for the same inputs you get the same output. Each call of GetBytes of .NET's Rfc2898DeriveBytes just returns more and more bytes generated by the algorithm whereas Java implementation needs to know the total output length upfront. So for 256-bit key and 128-bit IV we need to create PBEKeySpec with the length of 384 and split the result between key and IV:

KeySpec spec = new PBEKeySpec(passwordChars, 
    saltBytes, iterationCount, 256 + 128);
SecretKey secretKey = factory.generateSecret(spec);

byte[] data = secretKey.getEncoded();
byte[] keyBytes = new byte[256 / 8];
byte[] ivBytes = new byte[128 / 8];

System.arraycopy(data, 0, keyBytes, 0, 256 / 8);
System.arraycopy(data, 256 / 8, ivBytes, 0, 128 / 8);		

Note: All the Java stuff was tested only with Android.

Multilevel Geomipmapping Program + Sources Released

There's been a few requests for source code of Multilevel Geomipmapping terrain rendering. So I'm doing this now, finally. It has not been touched since 2008 but it compiles fine in current version of Lazarus. I tested it in Windows only but in 2008 it had run in Linux and FreeBSD as well. Unfortunately, not all the test terrain data could be included because of their massive size.

You can find more info in included Readme and previously linked article. Note: release archive is in 7z format to get smaller download size.

  Multilevel Geomipmapping
» 92.1 MiB - 97,698 hits - May 5, 2014
Terrain renderer using OpenGL. Includes Object Pascal source code, binaries, and test data.

  • Small terrain 2k x 2k
  • Multilevel tree nodes and wireframe display

Deskew Tool Version 1.10

New version of Deskew command line tool is ready. You can find general info about Deskew here Deskew Tools.

Change List for Deskew 1.10

  • TIFF support now also for Win64 and 32/64bit Linux platforms
  • forced output formats
  • fix: output file names were always lowercase
  • fix: preserves resolution metadata (e.g. 300dpi) of input when writing output

Continue reading