Thumbnail creation library (Epeg as a translator?)

1

I think we all agree that thumbnailing a folder full of images is a usefull feature of an OS.

All OS’es can do it. Even Beos/Haiku can do it when using Tracker.NewFS3.0. But it’s slow… way to slow to be usable. Windows is lighning fast at creating thumbnails of a lot of pictures. Beos needs this speed too!

The current tracker uses the default JPEG translator for reading all image data before it resizes the image to a thumbnail. The problem with the JPEG translator is it doesn’t provide the possibility to read pixels at specific coordinates. So you can’t say give me each 5th pixel on coordinate X & Y to speed up reading. Therefore we need a new translator.

I’ve come across a library for resizing JPeg files at high speed. It’s called EPEG and is developed for Enlightenment, a linux windowmanager. I’ts open source and only depends on the Jpeg lib (wich is freely available) as far as I can see.

I know how to program in C++ on beos, but I know nothing about make files and gcc stuff. I only know how to use use BeIDE, sad isn’t it??! :wink:

As far as I can understand the epeg source must be compiled to libepeg.so (for instance), before you can use it’s features. Is anyone here capable of porting Epeg to Beos, or at least give me some info on how it should be done?

If we could make a translator version of this library we should have fast, or at least faster, thumbnailing! I don’t know if this idea is even viable, or has already been tried, so let me know it’s good or bad!

Some useful links:
Epeg home: http://www.enlightenment.org/Libraries/Epeg/
Jpeg lib: http://www.ijg.org/

Don’t have the link to the epeg sourcecode right here… I’ll post it when I get home.

2

Making use of libepeg sounds like a good idea. The question is if it should replace the current jpeg translator or be a separate "thumbnail translator".

From what I can see in <be/translation/TranslatorFormats.h>, there is no standard way to tell an image translator that you want the image to be resized during translation. For this reason, I guess the translator would have to be distinguishable from the others, so that the app doesn’t send a parameter that the translator ignores, and you end up with a full size “thumbnail”.

Of course, since epeg only scales jpeg images, you also want a png thumbnailer, a tiff thumbnailer, a gif thumbnailer and so on. Maybe thumbnail would be a separate mime from image? (thumb/jpeg, thumb/png) Or maybe all of the existing image translators should be extended with a requested size argument? The jpeg translator would then be based on libepeg and probably be faster than the others, unless the resizing code from epeg is simple enough to implement in all Haiku image translators?

3

I think it should exist as a separate Translator. I also don’t foresee any problems of selecting the right translator since Tracker can use whatever translator it desires (at least i hope so :-)). Or am I missing something?

And about the other formats: it isn’t a big problem (for now) if they are thumbnailed the “old” way, since most images are jpeg… Like all digicam images.

Somehow I also have the feeling that this lib can easily be ported

4
Mauro wrote:
And about the other formats: it isn't a big problem (for now) if they are thumbnailed the "old" way, since most images are jpeg.... Like all digicam images.

Isn’t more easier to extract thumbnails from exif?

5

I had a look at the epeg source, and now I’m not so sure anymore that using this library is such a great gain. Don’t get me wrong, it’s blazing fast, but I was kind of disappointed to learn that all it does is; use libjpeg to decode original image, do it’s own scaling routine, use libjpeg to encode thumbnail image. I was kind of expecting the scaledown to be integrated in the decoding stage, and that this would make all the difference, but I guess that’s not even fully possible because of correlation between pixels.

Probably it’s better, after all, to use normal translators for input and output and implement the same simple scaling algorithm as libepeg uses in between. What makes epeg fast, could be that it’s only meant for thumbnailing, it’s not a general image scaling tool. Because it knows it will ever only scale things down, it does so simply by skipping pixels, and also it does so in-place. Here’s the general algorithm:

int x, y, i;
unsigned char *src, *dst, *row;
dst = buffer;
for(y = 0; y < new_height; y++) {
  row = buffer + (((y * old_height) / new_height) * bpp * old_width);
  for(x = 0; x < new_width; x++) {
    src = row + (((x * old_width) / new_width) * bpp);
    for(i = 0; i < bpp; i++) dst[i] = src[i];
    dst += bpp; // bytes per pixel
  }
}

You’ll have a hard time coming up with a faster algorithm than this :wink: The only thing I guess could be slightly improved is that if your intermediate format always uses 8, 16 or 32 bits per pixel, you can get rid of the bpp part by using three different versions of the function, where src, dst and row are pointers of unsigned char, short and long, dependent on bpp. It probably doesn’t make all that big a difference, though.

I don’t know if the current Tracker uses an inferior resize algorithm, or if it’s the overhead of the translator API that slows it down. Maybe libepeg initializes and calls libjpeg in a simplified way? What I’m sure of, is that it does use the same library as the jpeg translator, and it does make this library decode the full image before scaling takes place.

6
bogomipz wrote:
I don't know if the current Tracker uses an inferior resize algorithm, or if it's the overhead of the translator API that slows it down. Maybe libepeg initializes and calls libjpeg in a simplified way? What I'm sure of, is that it does use the same library as the jpeg translator, and it does make this library decode the full image before scaling takes place.

No, if you want speedy thumbnailing of digital camera images the solution is to read the existing thumbnail that is already stored in the exif file as metadata (you know, the one you see on your camera when looking at your pictures). Just as zuMikkebe said… Reading the full picture and resizing it will always be slower.

7

I didn’t know the EXIF data also contained a thumbnail. But there are 2 issues when relying on exif-thumbs:

  • not all jpegs contain a thumb in exif data.
  • thumbnails are very little. Therefore you can’t view all thumbs at a decent size. I often find the thumbnails made by windows too small.
Quote:
Don't get me wrong, it's blazing fast, but I was kind of disappointed to learn that all it does is; use libjpeg to decode original image, do it's own scaling routine, use libjpeg to encode thumbnail image. I was kind of expecting the scaledown to be integrated in the decoding stage, and that this would make all the difference, but I guess that's not even fully possible because of correlation between pixels.

I think the scaledown is integrated in the decoding stage, since not the entire image is decoded. It seems epeg only decodes the image partly to retrieve some "raw" image data. Check this:

It makes use of libjpeg features of being able to load an image by only decoding the DCT coefficients needed to reconstruct an image of the size desired
source

I’ve quoted this source for giving an impression about the possible speedups.

I have something like 1300 photos, most of which are 2560x1920 in size. By any stretch this isn’t small. When making thumbnails of these images, under Nautilus on my machine the process would take about 1 second per photo… …Epeg can thumbnail at about 25 thumbnails per second. Compared to the original speed of Nautilus.

Som usefull reading: http://www.systhread.net/texts/200507epeg1.php

I am sure it will be usefull for Tracker when it comes to creating thumbnails.

8

Aha, now I see. libepeg does tell libjpeg to decode a smaller image, but because the result will not be the exact size it needs, it has to skip some pixels to get the final thumbnail.

There are still lots of questions to ask, and approaches one could take. For instance, should the thumbnail be cached (in an extended attr) or does it have to be performed each time to fit the thumbnail size the user has currently selected for the view? Perhaps it should be cached at "maximum thumbnail size" and a final scaledown be performed when viewed? Does the size of the EXIF thumbnail differ between image files, and is it always too small to be used for large thumbnail view settings? What should the architecture look like to make thumbnailing feasible for all image formats? I believe other formats, like png and to a lesser extent tiff, is very common for screenshots and wallpapers. Should a thumbnail option be added to the standard image translators?

9
bogomipz wrote:
For instance, should the thumbnail be cached (in an extended attr) or does it have to be performed each time to fit the thumbnail size the user has currently selected for the view?

TrackerNewFS caches the generated thumbnail for future use. This should always be done in my opinion. So all the functionality is already supported by tracker.

10
Mauro wrote:
bogomipz wrote:
For instance, should the thumbnail be cached (in an extended attr) or does it have to be performed each time to fit the thumbnail size the user has currently selected for the view?

TrackerNewFS caches the generated thumbnail for future use. This should always be done in my opinion. So all the functionality is already supported by tracker.

Yes, I guess a large thumb (300px in biggest direction?) should be cached, then possibly scaled down when viewed.

I think the biggest question is how thumbnailing should relate to the use of translators. I kind of like the idea of a decode size parameter to the translator. You ask for a specific size or max side length. If you get something bigger it means that the translator for the format in question is unaware of the option, so you scale it down yourself.

The thumbnail logic could go into a library. This would include how to look for an already created thumbnail, and to create and cache it otherwise.

11

I wrote a simple test program which takes a directory as argument and creates a sub-directory (thumbs001, thumbs002, etc.) with scaled down versions of all the image files in the original dir. This actually took me a few days, being my first attempt at writing anything for the Be API (and I’m not particularly fluent in C++ either).

The program loops over all files found in the directory, skips those without mime super-type “image”, sends the files to decoding with whatever translator fits, scales the result down to 300px in largest direction, and writes a JPG file in the thumbs directory.

I ran this test four times with the R5 translator on a directory containing 40 jpeg images, all 1280x1024 in size. Then I dropped the Haiku translator in ~/config/Add-Ons/Translators and ran the exact same test four times with that. As you can see from the table below, the Haiku translator was about 4.5s slower than the one from R5 (I believe the “user” time is the interesting one).

The next thing I did was to add five lines of code to Haiku’s jpeg translator source, compile that and drop it in my Add-Ons directory. The patch is shown at the bottom of this post. What it does is tell libjpeg to scale the image with a factor 1/4, plus it sets a couple of other settings that I’m not really sure have any effect because I didn’t check their defaults. With this translator, the execution time dropped by nearly 8s compared to the unpatched version. This is of course a hack which is not usable, merely a test, as all programs will now see jpeg files in 25% of their real size!

These results might not look all that impressive, but keep in mind that this was on a rather slow laptop. For reference, I ran a similar test with the sample program that comes with libepeg using a for loop in bash to convert the 40 wallpapers. The difference between the epeg binary and the hacked translator was smaller than I had expected. (Epeg was surprisingly easy to get going on BeOS, btw. Just make sure you have libjpeg installed first, it can be taken from the jpeg translator directory in the Haiku source tree.)

# time ./thumbs ~/wallpapers real user sys
.
R5 JPEG translator 19.694s 15.485s 1.520s
17.839s 14.977s 1.145s
17.315s 14.927s 1.088s
17.794s 14.974s 1.108s
.
Stock Haiku translator 21.485s 18.452s 1.184s
21.551s 18.418s 1.176s
20.902s 18.351s 1.176s
20.926s 18.348s 1.162s
.
Scale factor hardcoded to 1/4 14.159s 11.049s 1.154s
12.855s 10.635s 0.937s
12.006s 10.537s 0.903s
12.134s 10.551s 0.920s
.
The epeg cli binary 8.191s 6.294s 1.151s
9.111s 6.310s 1.265s
8.238s 6.286s 1.175s
8.395s 6.308s 1.194s
--- src/add-ons/translators/jpeg/JPEGTranslator.cpp copy Sat Jan 20 20:50:17 2007 +++ src/add-ons/translators/jpeg/JPEGTranslator.cpp Wed Jan 24 22:51:50 2007 @@ -1372,6 +1372,12 @@ break; } } + + cinfo.scale_num = 1; + cinfo.scale_denom = 4; + cinfo.do_fancy_upsampling = false; + cinfo.do_block_smoothing = false; + cinfo.dct_method = JDCT_IFAST; // Initialize decompression jpeg_start_decompress(&cinfo);
12

Is there any update in this code