Digital Comic Downloader discontinued

Update: 2008-04-25 17:15

I've decided to take down Digital Comic Downloader. I'm all about digital distribution as where comics need to go, so I'm afraid of making it less palatable to companies like Marvel (and others in the future) by causing any cold feet... As I said when I launched DCD, the purpose is not piracy but I have to admit that it is capable of enabling it. (It's probably a bit unrealistic for me to think that because the images are not perfect that users would never choose to spread them around.)

However, I'm going to start working on a new application able to view comics in a nice full-screen mode, but not able to save files to disk. My hope is that this approach will be viewed as a good balance between user needs and corporate interests.

Update: 2008-04-24 02:48

Fixed! I was staring at the logs trying to figure out where the new dynamic "secret" value came from when I realized that I was looking at an MD5 hash. After that it was easy to run the viewer SWF file through Sothink SWF Decompiler to see how it was generated. They concatenate the issue id, your session id, and a salt value of "tcbsqudqm23rue4pnvcja86tl4" with pipes and take the md5sum of that. The download links are now updated to the new versions.

Update: 2008-04-23 10:55

Broken again. That was pretty fast! I'll see what I can do at lunch, otherwise any fix will be later this evening.

Update: 2008-04-23 02:27

Fixed! Marvel changed issuedata.php to only return metadata when you POST (not GET) this to it:
onLoad=%5Btype%20Function%5D
&secret=r586io7qu60mops19em15688j6
&session%5Fid=[sessionid]
&dc%5Fid=[issueid]

Perhaps they will change the value of "secret" a lot to make things annoying? Anyway, the download links below have been updated with the new versions. Marvel, you are welcome to download these too. :)

Update: 2008-04-22 18:55

Whatever Marvel did on their side seems to have broken their own viewer, too. It works fine in Firefox, but as of today doesn't work in Internet Explorer 7 under Windows Vista for me. Anyone else have the same problem? Also they were the first ones to download the DCD source code so there's no telling what further changes might be made on their side. If the current avenue becomes impractical I do have some ideas for automated screen captures... But I'd rather not go down that road.

Update: 2008-04-22 13:03

Broken again. issuedata.php is no longer returning the necessary metadata. If you try it and get a message saying that "such and such directory exists, do you want to delete it?", say no.

I won't be able to troubleshoot this change until I get home tonight, so I've decided to release the source code. Get it here: source code. If you can make it work again faster than me, go for it!

Update: 2008-04-21 18:07

Welcome to anyone coming from this week's Lying In The Gutters! Marvel downloaded the new version described below just minutes after it was posted. So if you use it and it stops working suddenly, please be patient while I research a fix. For now though, enjoy! (To be fair, the issue earlier is suspicious but still might've been a coincidence. No one from Marvel has contacted me, so there isn't necessarily a conspiracy afoot.)

Update: 2008-04-21 16:52

Nevermind, it was an easy fix. The MDCU server started rejecting requests from browsers with a blank agent-string. I've set a proper one in the program and the download link below has been updated. I am committed to providing an alternative to Marvel's unusable web viewer.

Update: 2008-04-21 16:42

Hmm. Someone from a Marvel IP address downloaded DCD this afternoon... And now it can't communicate with the server! I'll see what I can do later tonight.

Original:

I'm pleased to announce the availability of Digital Comic Downloader 1.0.

What it is

If you're a subscriber to Marvel Digital Comics Unlimited, you know how awful the official web viewer is. It displays pages, sure, but it is basically useless for reading. Digital Comic Downloader brings the selected comic down to your hard drive, allowing you to choose which viewing software to use. This makes reading issues a pleasant experience, letting you make the most of your subscription. It can even combine the separate text and art layers of newer comics into flat files!

What it is not

Digital Comic Downloader is not meant for piracy. The issues that you download should be the ones that you're actually about to read (and later delete). Marvel has been very kind in not using encryption or DRM and that needs to be respected.

Further, the downloads produced are not perfect copies, so even if you had a mind to infringe copyright, what you've got is useless for archiving... But great for reading.

Screenshot

Requirements

- An active, paid Marvel Digital Comics Unlimited subscription
- Windows 2000/XP/Vista (only tested on Vista so far)
- Microsoft .NET Framework 3.5
- Adobe Flash plug-in
- A standalone comic reader like CDisplay or any other image-viewing application

Download

Downloads are disabled. Thanks for everyone's support!

How to use

This is early, unsupported software. Only rudimentary error-checking exists. However: Once you've installed the program and any prerequisites, click on the Digital Comic Downloader entry in your Start Menu to begin. The fields should mostly be self-explanatory... Fill them in and click Download and you're off to the races.

One thing that might might confuse is the "Issue ID" field. To find this number, access the Marvel Digital Comics Unlimited catalogue as normal. When you pick something you want to read, mouse over the "OPEN" button and look at your web browser's status bar:

As you can see in the screenshot, Astonishing X-Men #9 is Issue ID number 1454. Use that when filling out the fields in DCD.

Feedback

Please leave feedback with any problems you encounter and any ideas you have for further development.

Here are features that I already plan to implement, so don't worry about suggesting them:
- Auto-compress the download folder into a zip/cbz. Probably in the next version. No rar/cbr support though, because there aren't any free libraries for creating rar archives. (only for decompressing them)
- Proxy support (at the moment it uses whatever Internet Explorer is set to... I think)
- Store the metadata that Marvel provides. There's all sorts of information like whether a page is a double spread, an advertisement, what year the series started, etc... This could be useful for future comic reading software.
- There will not be a full-series download function. I don't want to position this application as anything but a "hey, I want to read this issue but the official viewer sucks" sort of helper. So don't ask, thanks!

Credits

SWFToImage library by ByteScout, from http://www.bytescout.com/swftoimage.html
MagickNet library by ?, from http://midimick.com/magicknet/

Read the full article

Image manipulation progress

Some progress on the comic downloader.

I've got programmatic image manipulation largely done. The process is basically this: Render the SWF text layer at twice the size of the JPG background layer, then make it transparent and scale down to fit. (That makes the output quality a bit better.)

To test things out I used the command-line ImageMagick tools:

convert -fuzz 115 foreground.png -transparent "#ff00ff" -resize 1000x1500 foreground-transparent.png

composite foreground-transparent.png background.jpg final.png

Then I added the MagickNet DLL (a .NET wrapper for the ImageMagick libraries) to my project and rewrote the steps in C#:

MagickNet.Magick.Init();
MagickNet.Image imgForeground = new MagickNet.Image(Application.StartupPath + @"\foreground.png");

imgForeground.ColorFuzz = 115;
MagickNet.Color imgColour = new MagickNet.Color(255, 0, 255);
imgForeground.Transparent(imgColour);
Size imgSize = new System.Drawing.Size(1000, 1500);
imgForeground.Resize(imgSize);
imgForeground.Write(Application.StartupPath + @"\foreground-transparent.png");

MagickNet.Image imgBackground = new MagickNet.Image(Application.StartupPath + @"\background.jpg");
MagickNet.Image imgForegroundTransparent = new MagickNet.Image(Application.StartupPath + @"\foreground-transparent.png");

imgBackground.Composite(imgForegroundTransparent,0,0,CompositeOperator.AtopCompositeOp);
imgBackground.Write(Application.StartupPath + @"\final.png");

MagickNet.Magick.Term();

Much to my surprise, it worked! Here's what the input and output images look like:

Foreground


Transparent foreground (note: some browsers can't render transparent PNG files correctly)


Background


Final


It's not perfect, because if you zoom in 1200% you can see some purple left on the edges of the speech balloons:



However the goal is not in creating pristine archive copies for hoarding, but temporary copies for convenient reading. So it's fine. :)

Next up is to see if I can replicate the full MDCU login process. The current method involves: logging in normally, opening a comic, then viewing the HTML source of the page to get the session id and issue id, then pasting those into my program. Not terribly convenient.

Read the full article

More about the comic download tool

I expected to release a downloading tool for Marvel Digital Comics Unlimited at the end of last week but ran into some last-minute troubles.

The comics online at MCDU exist in two different formats: Older issues that appear to have been scanned from paper, and newer issues that appear to have come directly from a digital workflow. The old ones are plain JPG files that are easy enough to use in a third-party application. The new ones however, are JPG files of background art with SWF files containing speech balloons layered on top.

To convert the SWF text layer into an image I've been using ByteScout's SWF To Image library. It's a decent, free tool but there have been some hiccups. The text SWF files have an overall movie background colour of white, or #FFFFFF. So if you convert them as-is, you get white speech balloons on top of a white background:

That's no good, because if you deleted all of the white and put it on top of the background you'd have this:

So as a solution I tried creating my own SWF that would load the JPG and text SWF in layers and then convert all that into a single image. It worked... Sort of. SWF To Image executes and captures an image too quickly for the loadMovie commands in my custom SWF to finish loading Marvel's SWF and JPG. The only time it worked was when I used an example VBScript with a MsgBox prompt as an interruption, or in my actual program by writing the final image to disk several times in a row. That's not a hack I'm willing to live with.

I messed around with various ways of loading content in Flash with events that determine whether or not the content has finished loading but in the end I couldn't reliably render the correct final image. For a different approach I looked into the SWF format itself. This reference to the SetBackgroundColor command being an RGB value set in every movie got me thinking... If I altered the downloaded SWF files to use a different background colour, it could easily be replaced.

By changing this series of bytes "43 02 FF FF FF" in the header of the text SWF to "43 02 FF 00 FF", I'm left with exactly what I need:

Now that the background isn't white, purple is deleted instead, leaving this on top of the background:

However this new method has not been done programmatically yet. (Only with a hex editor, some screenshots, and Photoshop.) There won't be an issue replacing bytes representing the background colour, but going on to cleanly remove that new purple background may require some fine-tuning.

More to come...

Read the full article

Ikaruga on Xbox Live Arcade

Ikaruga has its detractors, but I am extremely grateful to Microsoft and Treasure for supporting monitor rotation and the 16:10 aspect ratio. There's nothing like playing a vertical shmup the way it was meant to be played:

Read the full article

Marvel Digital Comics Unlimited security

This is something I discovered a few months ago but never got around to writing about here. Marvel Digital Comics Unlimited is a subscription service that allows one to read scanned versions of Marvel comic books online. The issues that one can read are never current with those on the shelves, but I suppose Marvel had to make some concessions to retailers.

It's a decent enough service, but the Flash applet used for reading the books in your web browser is nigh unusable. So just after Christmas I started working on a Firefox extension that would allow me to read the books in an offline reader like CDisplay.

In researching how the Flash applet communicated with Marvel's server, I discovered that although an authentication value was being passed with every request, it was being completely ignored. The upshot being that a request for an image file like http://www.marvel.com/dotcomics_issues/ASM093_1963/hi_res_col/02.jpg?stdi=ckfq35k8nfep915892v1nsift2dgr didn't actually require the "stdi" variable to be passed, effectively making every comic open to subscribers and non-subscribers alike.

I decided to inform Marvel, but I had no interest in wading through a phone tree trying to explain security vulnerabilities to receptionists. Unfortunately, Marvel's contact information page is fairly useless unless you want to book Spider-Man for your next corporate function. So I contacted Rich Johnston of Lying In The Gutters, a long-running comic book rumour column.

In the January 7th, 2008 edition of LITG (under "Marvel Comics Unpaid"), Rich reported my findings and asked Marvel for comments. Apparently they never responded, but the security hole was fixed that very day. My guess is that Marvel was aware of the issue but hadn't pushed a fix out to production yet. I know how that can go sometimes.

Anyway, the reason I got around to writing this is that I expect to have a proof-of-concept of my downloading utility later this week. It won't be a Firefox extension as I'd originally hoped, but I will try to make it as convenient as possible.

Read the full article