Queen Rania focused primarily on the need for native grassroots movements in the Arab world, stating that civic engagement is an essential element of societal reform—changes so desperately needed in this region, where poverty and corruption are rampant. She stated that it is the responsibility of all citizens to take an interest in developing and improving their own country. Calling on Arabs to “look up” and think of others, she cited the example of one of AUC’s homegrown NGOs, Alashanek Ya Balady, which is heavily involved in promoting sustainable development in Egypt’s poorest neighborhoods. The efforts of AYB and dozens of other organizations constitute a laudable homegrown effort to reform Egyptian society, and they have had lots of success so far.

The queen then went on to decry the current state of civic laziness in the Arab world, saying that many Arabs confuse the responsibilities of “citizenship” with “sitizenship,” opting to sit still, sit back, and complain rather than take their own initiative to change society. This is not because Arabs are disconnected, emotionless people—far from it. “We’re all passionate about food, family, football, and Filistin,” she said, but those passions need to be refocused on bringing about change instead of complacently complaining about the backwardness of Arab governments and societies.

She developed this theme by responding to a series of preselected questions that followed her main motivational speech, declaring that universities and academia are responsible for remaining engaged with society and not remain isolated in academia. Too often the educated class remains cynically aloof from the rest of the world and fails to contribute real change or instill the culture of civic engagement in their pupils. She also noted the role of Islam in developing this culture of engagement, saying that Islam inherently advocates civic reform and grassroots movements. She called on the audience to “recapture those compassionate values of Islam” that have been hijacked by Western media and use the power of faith to help improve society.

Essentially, she concluded, if we want to see any reform and improvement in the region, the onus is on citizens. Change in the Middle East cannot come from governments, since they are inherently inefficient. Real change can only come about with a change in attitude and an increase in engagement. Regular citizens are the driving factor for this change.

In the end it was a pretty motivating and inspiring speech, and I agree with most of what she said. Real, lasting change in the Middle East will need to come from some type of bottom-up popular movement, especially when the reforms so desperately needed reduce the power of the ruling class. However, repressive governments throughout the Arab world (including Rania’s Jordan) severely limit what progressive movements can do, and Middle Eastern dictatorships last for decades.

For example, Egypt’s current president, Hosni Mubarak, has been in power for almost 30 years. He ran the country uncontestedly until 2005, when, after pressure from George W. Bush, Mubarak amended the constitution to allow candidates from parties other than the ruling National Democratic Party (NDP) to run in presidential elections. The first “free and fair” elections were held that year. Despite a somewhat large grassroots political opposition movement, Kifaya, Mubarak won handily and his opponent, Ayman Nour, was arrested and sentenced to prison for five years. Today Kifaya, a native grassroots movement, is detoothed and powerless. Good thing civic engagement worked…

The next Egyptian presidential elections are slated for 2011, but since Mubarak is getting on in years (he’s 81!), he’s probably not going to run. Instead, it seems that his son, Gamal Mubarak, will inherit his father’s place. Sure, he’ll go through the farce of the election process, but the NDP politicos and thugs will pretty much guarantee his victory.

Understandably, there is a growing anti-Gamal movement in Egypt, and it’s even homegrown and grassrooty. It got a huge boost last week when Mohamed ElBaradei, former head of the IAEA and winner of a Nobel Prize, returned to Cairo after a decades-long pseudo-exile. He has announced that he would consider running for president in 2011 if circumstances allowed for it. He already has a large base of support and even an official-ish campaign website. American media is picking up on him, too. He has a lot of international clout and could present a real threat to the Mubarak dynasty. He might have the potential of becoming the Egyptian Obama—“Yes we can!”

But (and this is a pretty big but…), he can’t even legally run—he’s not constitutionally permitted to run for president. For him to become president, Mubarak would first have to amend the constitution to open the field for more opposition parities. Then ElBaradei would actually have to win in elections run by the NDP. Yeah. Good luck there.

While the possibility of an ElBaradei run has Mubarak somewhat scared, the cards are really stacked against him. This growing popular movement faces the impossible task of forcing a constitutional amendment. It’s a native movement, just like Queen Rania wants, but it is severely limited. Egyptians remain hopeful, but sadly, Gamal will most likely take over in 2011.

While Jordan, as a monarchy, doesn’t face these issues of presidential elections, it has its own slew of problems with citizenship. Although Rania herself is a Palestinian, the Jordanian government regularly withdraws Jordanian nationality and rights from Palestinian refugees. How can native, grassroots Jordanian organizations reform when the government makes them stateless?

In her speech Rania exhorted AUCians: “Don’t wait for governments to change and reform. You must be the change you want to see.” Um. She’s in the Jordanian government. She’s the queen! She surely has some influence on the Jordanian political scene. Can’t she help the government change so that these popular groups can actually do something? Arab governments tolerate, even embrace, NGOs like AYB or the queen’s water projects in Jordan—they love this type of civic engagement. But heaven forbid these civically engaged movements threaten their despotic power, though.

Can real governmental and societal change happen in the Middle East solely through “civic engagement.” No way. Arab governments must make changes if any popular movements want to make substantial change.

So, while Queen Rania’s speech was inspiring and motivational on the surface, it was full of kalaam fadi—empty words.

Normally to run Flashbake consistently you need to set up a cron job. While OS X is built on Unix and has cron, Apple recommends using launchd and property list (plist) files to run system agents and daemons. flashbakectl is a handy little script that loads and unloads a plist for you.

Before working on your project, run flashbakectl -l to load the plist and start the daemon, which will commit your unsaved changes every 15 minutes (or whatever you set it to). When you’re done for the day, run flashbakectl -u to stop the daemon, saving your computer from unnecessarily running Flashbake ad infinitum.

flashbakectl only works on Mac OS X. You can get it at GitHub. Enjoy!

Flashbake includes several plugins for adding recent tweets, weather, the current time zone, and other random information. There’s even a plugin for the Banshee music player for Linux. There’s nothing for iTunes, however, which is unfortunate since I’m always listening to something when I write or code.

So I hacked together a little plugin for Flashbake that uses AppleScript to get the current track information from iTunes and add it to the commit message. It’s admittedly a “frankenscript” and only works on Mac OS X (since it relies on AppleScript), but it works great.

You can get it at GitHub. Enjoy!

Google Voice, the Google-ized incarnation of GrandCentral, is a fantastic service that aims to become your virtual phone switchboard. It gives you a free phone number that can receive regular phone calls and route them to any other actual phones you have connected to your account. Powerful stuff.

Unfortunately, though, its forwarding abilities are limited to US phones. Using some VoIP magic, though, you can create a semblance of international forwarding and get free (or nearly free) phone calls to the US while abroad. If you’re not in a foreign country, you can harness the same VoIP magic to get a nearly free phone service.

The Gizmo Project—background

Gizmo is normally a VoIP provider that lets you make free (or super cheap—something like $0.019 a minute) phone calls. When you sign up for an account you get a special phone number in the 747 area code as your VoIP/Gizmo username. While any phone on the Gizmo network can call your 747 number for free, regular phones can’t connect to it.

Gizmo offers a Call In service that lets you buy a phone number in most US area codes (or one of dozens of countries), which then lets you receive phone calls from standard phones. Call In numbers start at $35 a year (or $12 a year for three months), but prices can be higher depending on demand.

Gizmo touts itself primarily as software—it provides a “soft phone” program that you run on your computer. As long as the program is open you can make and receive phone calls (much like an IM program or Skype) using a microphone and your computer’s speakers or headphones.

Gizmo without a computer

It’s impractical to keep your computer on all the time and it can be awkward to use your computer as a phone. You can get around this limitation by buying an ATA adapter—a little box that plugs into your network with the sole purpose of running phone services. It’s essentially a hardware version of the Gizmo soft phone.

Fancy corporate VoIP phones (like the ubiquitous Cisco ones) have ATAs built in (kind of. The real ATA is somewhere on the network letting these computer-phones connect to it). You don’t need a fancy VoIP phone, though. Standard consumer ATAs let you plug regular phones directly into the adapter.

After configuring the ATA with your Gizmo information you can make and receive calls using your Call In number. Rather than pay $40+ a month for regular phone service, you can have a fully featured phone that only costs $35 a year plus <$0.02 a minute.

Before Google Voice

For the past three years we’ve been using Gizmo as our full-time phone. We bought a Linksys/Sipura ATA (our model, the SPA1000 is no longer manufactured), got a $10 phone from Target, and bought a Utah county 801 Call In number. The phone worked perfectly. I could plug the ATA in to any internet connection and get cheap/free phone service.

Our system even works (mostly) in Egypt. I have two phones on my desk: the $10 Target phone plugged into the ATA (which is plugged into the router) and a 20 EGP neon pink phone (plugged into our Egyptian phone line). Family, friends, and unsuspecting telemarketers can reach us at our Utah number and pay only what it costs them to call an 801 number.

Our current phone set up

The only problem with the system our system in Egypt is a bizarre limitation with Egyptian (or at least Link.net’s) internet infrastructure. We don’t have any bandwidth issues when someone calls us, but when we call out, the connection drops within the first ten seconds of the call 90% of the time. To get around this, we used some SkypeOut credits—we’d call someone in the States with Skype (using my computer), tell them to call us on our Utah number, hang up, and wait for their call.

This worked when we called actual people, but doesn’t work when calling banks, insurance companies, airline companies, or anything else with a phone tree—phone trees can’t call you back. SkypeOut works for those, but it’s more expensive than Gizmo.

Google Voice changes all this.

Enter Gizmo Voice

Google and Gizmo have joined up to let you hook Google Voice directly into your Gizmo account. Rather than buy a Gizmo Call In number, I can use my free Google Voice number with my Gizmo Account. After following Gizmo’s instructions on connecting the two accounts, now when people call my Google Voice number, the call is routed to the normally inaccessible 747 Gizmo number, which is already associated with my ATA box.

This means I can stop paying $35 a year for my Call In number. The only thing I pay for is the phone use itself. Gizmo just changed their phone rates for users using Google Voice—apparently all calls under three minutes are free, while longer phone calls follow their normal low rates.

Additionally, now that I can have my Gizmo phone connected to my Google Voice account, my bizarre issue with calling out on Egyptian internet can be solved. In order to call people and have your Google Voice phone number appear on their caller IDs, you need to use Google Voice as an intermediary. You type in the number you want to call on their website and they’ll call one of your linked phones. When you pick up, your phone will start dialing the outbound number. Since Google calls my Gizmo phone now to make outbound calls, Link.net considers it an inbound call and it doesn’t get cut off.

So now, Gizmo combined with Google Voice gives me free short calls and cheap long calls to the US and a free US number that can replace my Gizmo Call In number. Everything works both in the States and internationally. It’s a nearly perfect system.

Still untested Now tested…

In theory, since I have my Gizmo and GV accounts linked, my Google Voice number should show up on the recipient’s caller ID when I call out with my US phone, circumventing the need to use the Google Voice web site as the middleman. I can’t test it, though, since my internet connection won’t let me make any outbound calls with my ATA. I’ll keep trying over the next week, since I can make like 10–15 second calls 10% of the time.

It works! My Google Voice number shows up, just like it should…

Update

Google has acquired Gizmo5, which hopefully means that the link between Gizmo and GV will be more permanent and more official. Awesome.

After almost a year of being in Egypt, we finally decided to go up to Alexandria today. We took the 9:00 AM train from Cairo and the ride went smooth until we were just outside the final Mahatat Misr station, where we were delayed for over an hour until pulling up to a platform.

Immediately after we got off the train we saw the reason for the delay. A train had rammed into the station, apparently at full speed, derailing the first three cars—the engine, the generator, and the first actual cabin. Part of the station itself was damaged, as was, ironically enough, a parked fire truck.

Click to enlarge

The train crashed at around 8 AM and was empty. By 12:30 (when we got there), Egyptian Railway repair crews were lifting the damaged cabin off the tracks.

By 6:00 PM, when we arrived back at the station to return to Cairo, the repairmen were working on lifting the engine and had apparently already extricated the generator car.

Since two platforms were out of commission, trains were shuffled around all day, causing major delays. Our train ride back to Cairo took over four hours instead of the usual two.

As far as I know, nobody was killed as the train was empty. I don’t know about injuries, since I arrived on the scene several hours after the actual accident.

Needless to say, it was exciting—in a macabre sort of way. I’ve read so much about Egyptian train crashes—I got to see the aftermath of one.

I’ll post the full set of pictures tomorrow on Flickr.

Update

I posted our photos of the accident at Flickr. They are licensed under a Creative Commons Attribution license, so you can use them however you want.

Nancy has also blogged about the accident at our family blog.

Additionally, the online Egyptian news site Youm7 has posted an update (in Arabic). According to their report, the accident was caused by a brake failure as the train travelled from the nearby Sidi Gaber train station. It was indeed empty, fortunately, and there were no fatalities. Two of the crew members were injured in addition to a vendor—since there are snack booths at the end of each platform, he must have gotten smashed. There are conflicting statements as to the cause of the accident. Some claim neglect and disrepair; others sabotage. I’m leaning towards neglect—those trains are ancient.

Basic usage

Initial set up

Download the most recent version of pdftk-php from GitHub and download and install pdftk on your server.

Unzip the download from GitHub and place the folder on your server. I’ve placed mine in a folder called pdftk-php.

Create a MySQL user and database and run the SQL found in /example/database.sql in a MySQL client (like phpMyAdmin) to create the sample database.

Example query in phpMyAdmin

Modify the information in example/_dbConfig.php so that the application can connect to your database.

$host = "localhost"; 
$username = "pdftk-user"; 
$password = "supersecure"; 
$db_name = "pdftk-php";

Browse to the example site (in my case, http://localhost/pdftk-php/example/index.php) and add some entries to populate the database a little.

Set up the script

Open pdftk-php.php and insert the full path to your working pdftk installation at the beginning part of the passthru() command near line 71. Here are some examples for different scenarios on server platforms:

// On a typical Unix-based installation
passthru("/usr/local/bin/pdftk ...");

// On Windows, with an absolute path
passthru("c:\pdftk\pdftk.exe ...");

// On Windows, with a relative path (useful if you place pdftk.exe in the server folder structure)
passthru("../pdftk.exe ...");

If you’re on a Unix-based server and don’t know where pdftk is, type one of the following commands, which should result in the absolute path to the program:

which pdftk
# or
whereis pdftk

In example/download.php verify that the path to the required pdftk-php.php is correct, near line 18. In the example, pdftk-php.php is located a directory below the example directory. If you like to store your included files elsewhere, make sure that you modify the require() path here.

pdftk-php.php needs to be able to write to a temporary directory on your server in order to create a temporary FDF file. This directory is specified near line 58, with the tempnam() function.

If you are on a Windows server you should already be able to write to pretty much any directory (I think… I’ve never worked with IIS permissions), so you should be good to go. If you are on a Unix-based server you’ll need to be more explicit with directory permissions. To make things easier, create a temporary folder on your server and give it write permissions:

cd pdftk-php
mkdir tmp
chmod 777 tmp

Then set the path in tempnam() to the new temporary folder.

// If at the same level as download.php
$fdf_fn = tempnam("tmp", "fdf");

// If one directory behind download.php
$fdf_fn = tempnam("../tmp", "fdf");

// You can also use an absolute path
$fdf_fn = tempnam("/Library/WebServer/www/pdftk-php/tmp", "fdf");

Set up the PDF

Create a fillable form in either Acrobat Professional or LiveCycle Designer, or use the included example PDF form. Give each field a unique and significant name so that you can work with the form more easily later on. You can modify field attributes by double clicking on the field using the Forms toolbar in Acrobat; in LiveCycle, use the Object panel.

Acrobat Form Field Options

LiveCycle Form Field Options

If you are using LiveCycle, you’ll need to save the final PDF as a static form compatible with Acrobat 7. pdftk doesn’t work with dynamic forms or PDFs from later versions of Acrobat.

LiveCycle Save Options

Connect PDF to script

example/download.php connects to your database, retrieves a row based on a passed GET variable, saves the data from the fetched row into variables, finally calling pdftk-php.php, which does the heavy lifting of creating an FDF file and injecting it into the PDF.

Starting at around line 30 the script assigns the fetched values to variables. Each of those retrieved variables needs to be paired with a form field in your PDF (near line 39). In a basic Acrobat form this is simple:

$fdf_data_strings= array('firstname' => $pdf_firstname,  'lastname' => $pdf_lastname, 'email' => $pdf_email);

LiveCycle tends to complicate the form names slightly. You can use pdftk from the command line to retrieve the official form field names. Run this command from the directory containing your PDF file:

pdftk form.pdf dump_data_fields > form-fields.txt

When you open the resultant .txt file you should see a report of all the fields

...
---
FieldType: Text
FieldName: form1[0].#subform[0].firstname[0]
FieldNameAlt: First name	
FieldFlags: 0
FieldJustification: Left
---
...

Use those long, hairy FieldNames in the $fdf_data_strings array, like so:

$fdf_data_strings= array('form1[0].#subform[0].#area[0].FirstName[0]' => $pdf_firstname, 'form1[0].#subform[0].#area[0].LastName[0]' => $pdf_lastname, 'form1[0].#subform[0].#area[0].EMail[0]' => $pdf_email, );

Finally, check the values of $pdf_filename and $pdf_original near lines 62 and 65.

Go to http://localhost/pdftk-php/example/view.php and click on the download links for one of entries. You should be prompted to download a PDF file, dynamically generated using pdftk-php.php. Success!

Advanced customization

Using checkboxes or radio buttons

$fdf_data_strings works great for text fields, but can’t handle radio buttons or check boxes. For that you’ll need to use the $fdf_data_names array near line 49.

NB: The logic for manipulating the form data in PHP and MySQL might be a little convoluted and could easily be optimized, but it works for clear demonstration purposes.

To demonstrate this how to do this, we’ll add a checkbox to our form and extend the database. Run this query in a MySQL client to add a couple columns to our table:

ALTER TABLE `users` ADD `option1` TINYINT( 1 ) NOT NULL, ADD `option2` TINYINT( 1 ) NOT NULL ;

Open /example/example.pdf in Acrobat Professional and add two checkbox fields named option1 and option2.

Ridiculously huge checkboxes

We need to modify our web form and the table that displays the data, just to make sure everything is getting saved to the database correctly.

First, make a couple changes to example/index.php After the section near lines 104–107, add

<p>
    <label for="option1">Option 1</label>
    <input type="checkbox" name="option1" value="1" id="option1" />
</p>
<p>
    <label for="option2">Option 2</label>
    <input type="checkbox" name="option2" value="1" id="option2" />
</p>

Then, up near the top of example/index.php after line 34, add this:

if ($_POST['option1'] == 1) {
    $option1 = 1;
} else {
    $option1 = 0;
}

if ($_POST['option2'] == 1) {
    $option2 = 1;
} else {
    $option2 = 0;
}

This checks the value of the submitted checkboxes and sets the $optionx variables to either 1 or 0, which fit into the TINYINT columns in our table. You could use actual text as well and set the columns to VARCHAR.

Change the SQL query from

$sql = "INSERT INTO users (firstname, lastname, email) VALUES ('$firstname', '$lastname', '$email')";

to

$sql = "INSERT INTO users (firstname, lastname, email, option1, option2) VALUES ('$firstname', '$lastname', '$email', '$option1', '$option2')";

Go ahead and insert some dummy submissions with boxes checked and unchecked to make sure everything is working.

Optionally we need to modify example/view.php to show the stored values. Add the following table header cells after line 29:

<!-- Already here --><th>E-mail Address</th>
<th>Option 1</th>
<th>Option 2</th>
<!-- Already here --><th>Download PDF</th>

In the while loop a few lines later, add this code:

<!-- Already here --><td><?php echo $user["email"]; ?></td>
<td><?php echo ($user['option1'] == 1) ? "Yes" : "No"; ?></td>
<td><?php echo ($user['option2'] == 1) ? "Yes" : "No"; ?></td>
<!-- Already here --><td><a href="download.php?id=<?php echo $user['id']; // the rest of this...

This is just PHP ternary notation, which essentially says that if the value of optionx is equal to one, echo “Yes,” otherwise, echo “No”.

Finally we need to modify example/download.php to handle our checkboxes. Like I said above, the $fdf_data_names variable handles checkbox and radio button data. In PDF forms, the two allowed values for checkboxes are “Yes” and “Off” (not really opposites, but oh well), so you’ll need to set variables accordingly. Replace $fdf_data_names = array(); near line 49, with this, which checks the values of optionx and sets $pdf_optionx to either “Yes” or “Off” and then defines the $fdf_data_names array appropriately:

if ($data['option1'] == 1) {
    $pdf_option1 = "Yes";
} else {
    $pdf_option1 = "Off";
}

if ($data['option2'] == 1) {
    $pdf_option2 = "Yes";
} else {
    $pdf_option2 = "Off";
}

$fdf_data_names = array('option1' => $pdf_option1, 'option2' => $pdf_option2);

And that should do it! Visit http://localhost/pdftk-php/example/view.php and download one of the forms. The checkboxes should populate perfectly.

Other types of form fields

Combo boxes and radio buttons act similarly to checkboxes. If you run the dump_data_fields command with pdftk again on a form with these more advanced options, you’ll see a few differences in the results.

FieldType: Text
FieldName: email
FieldFlags: 0
FieldJustification: Left
---
FieldType: Button
FieldName: option1
FieldFlags: 0
FieldValue: Yes
FieldJustification: Left
FieldStateOption: Off
FieldStateOption: Yes
---
FieldType: Choice
FieldName: favoriteColor
FieldFlags: 131072
FieldValue: blue
FieldValueDefault: red
FieldJustification: Left

You can see the “Yes” vs. “Off” values in our checkbox (called “Button” in PDF lingo). Drop down lists (“Choice” in PDF-speak) have multiple values, specified by you when you create the field.

Sample combo box properties in Acrobat

Radio buttons are hybrids. They are considered “Buttons,” like checkboxes, but can have custom values, like drop down lists.

Your final PHP script will need to take these different values into account and assign the correct values in the $fdf_data_names array.

Conclusion

You can do a ton with the pdftk-php.php class once you get it set up initially and get past the slight learning curve. If you have any questions, feel free to ask in the comments. If you find any problems, comment here or open an issue at the GitHub project page. Additionally, you can fork the project and contribute.

Good luck!

There’s only one problem: I don’t quite know what I’m writing it on.

At BYU I double majored in Middle East Studies/Arabic (MESA) and Italian—an odd mix of modern history, political science, and renaissance poetry. Most of my research focused on finding literary and historic connections between the Middle East and Italy. I looked at the role of Sufism in the birth of Catholic mysticism, especially with Jacopone da Todi and St. Francis of Assisi. I looked at the Young Ottomans and their reliance on Mazzini’s Giovine Italia ideology. I even presented a paper at a conference about the role of Mohammed in Inferno XXVIII in Dante’s Divine Comedy, connecting it to proto-orientalism.

Fun times

At the same time, though, there was an inherent conflict in my research interests. I wrote my MESA capstone paper on the media coverage of the 2006 Israel/Lebanon war, where no Italian literature was involved . I love media—I’m obsessed with the news, the internet, blogs, Twitter; anything shiny, new, and exciting.

I started my MA with the assumption that I’d have to choose one of these tracks—history or media. I dove headlong into the history track, writing a huge literature review on the history of the Italians in Egypt at the beginning of the 20th century. Large Italian communities in Cairo, Alexandria, and Ismailiyya sprang up after Napoleon’s 1798 invasion. Thousands of Italians were born and raised abroad in these cities, and many took part in the Egyptian nationalist movement, considering themselves more Egyptian than Italian, thus earning themselves the nickname mutamasirun (those who try to be Egyptian).

I was excited about this research until a new shiny thing took my attention away—Twitter. I started using Twitter more or less full time in January, which then introduced me to the large Egyptian Twitter community. Many of the Egyptian twitterers are also activist bloggers who have been arrested multiple times. I began to see the power and potential of the internet in political reform and change in the Middle East and switched research gears to focus on the Middle Eastern blogosphere. It was fascinating stuff, but I felt like I had turned my back on history Further complicating things, I had an amazing history seminar last semester that resulted in some awesomely fun archival research. I had a blast writing the paper. The primeval dichotomy of history vs. media reared its ugly head again.

Since I’m nearing the end of my MA, I’m looking at different PhD programs to go to once I’m done here. There are lots that look at bloggers and politics—even ones that look at bloggers and politics and the Middle East—but none do it with a historical approach (fancy that… blogs have been around for something like four years and it’s not history yet ). I sent out dozens of e-mails to different professors asking about graduate programs and my potential research. Most responses were confused; history!= new media.

However, one professor at Cornell responded positively. He studies media in modern Egyptian history, specifically in the time period of my Italian mutamasirun.

I think I may have found a way to bridge the history/media studies gap. Now I just need to figure out exactly how to do it.

Awesome.

So, sorry bloggers—I’ll keep following you as a tangential fascination, but my heart lies in history.

Since we’re both hobbyist graphic designers—I taught a couple print layout and design classes as an undergrad at BYU and have made several books at Lulu.com—we decided to layout and print each year of our blog, to keep for posterity.

A couple years ago Nancy attempted this with our smaller Jordan blog for a print publishing class she took at BYU. We spent the bulk of our time manually copying and pasting each post and the subsequent comments into a huge Word document. She then ran a long series of find/replaces to clean up the messy, inconsistent typography, and then finally placed it into Quark (that evil program). Through a series of unfortunate events, Quark crashed repeatedly and corrupted her file multiple times—she was lucky to get her first draft turned in for her final project (she got an A, though. Phew!).

I knew there had to be a faster, more efficient way to wrangle all the blog text, but this was back in 2006, before Blogger had an open API or options to backup a blog. Primitive, dark days indeed .

However, last year, Blogger introduced a fantastic new option—the ability to backup and export your entire blog, comments and all. Blogger spits out an Atom-formatted XML file that you can use to recreate your blog later on (or possibly import onto other platforms, like WordPress, I think). This was the key to simplifying the daunting task of collecting the text for our blog books. All we needed was a way to mangle the text in the XML file to create an InDesign-ready file.

So, I whipped up a semi-complicated Perl script that can parse an Atom-formatted XML file from Blogger and create a text file using InDesign Tagged Text to preapply paragraph and character styles. It also cleans up the typographic elements of the text, adding em and en dashes, removing empty paragraphs, etc. Additionally, it can add hidden index entries for each tag, essentially creating a barebones index for your book. And it only takes 10ish seconds to run on a large blog. It’s not perfect and could stand some good optimization, but it works.

Additionally, since InDesign tagged text works with, well, text, it won’t place your images for you. Instead it will insert the location of the image (the src=whatever.jpg of the img tags) in between curly braces { }. You’ll then need to manually place all the images later, deleting the braced text.

In the future, the script could be changed to output XML, which does let you include pictures, but you’d have to have all your images on your hard drive already. The script could go and download all the linked images, but it’s not really a good idea to place low resolution, web-optimized images in a print document. In our case we have high-res copies of all the pictures on the blog stored on an external hard drive, so we just have to go and find and place the images we want. It takes more time, but it makes better quality documents in the end.

Also, links are preserved as footnotes—all href="whatever.html"s show up as the footnote text.

How to use the script

First, download the script and its supporting files from Github. If you’re using Mac OS or Linux, make sure the main script file, format_for_id.pl is executable—type chmod +x format_for_id.pl at the terminal.

Next, make sure you have Perl installed on your system. If you are using Linux or Mac OS X, you’re good to go. If you’re using Windows, download and install Strawberry Perl for Windows. You can also use ActivePerl, but installing modules is a little more difficult.

The script uses several additional CPAN modules that you’ll need to install. You’ll need to use the CPAN shell to do so.

• On Windows with Strawberry Perl: open the packaged CPAN client in the Start Menu folder
• On Windows with ActivePerl: Good luck. There is a large repository of specially compiled CPAN modules for ActiveState, and reportedly there is a kind of CPAN shell, but I haven’t gotten either to work too well. Stick with Strawberry Perl. It’s better
• On Mac OS X: type perl -MCPAN -e shell at a terminal window
• On Linux: type sudo cpan at a terminal window

(If it’s your first time running the CPAN shell you’ll be asked to configure the installation environment. Choose the option to automatically configure everything.)

Once everything is set up and you see the cpan> shell prompt, type install Package::Name (eg. install Date::Format) for each of the dependent CPAN packages listed at the beginning of format_for_id.pl.

Log in to your Blogger Dashboard and export your blog as an XML file by going to Settings > Basic > Export blog. Place the XML file in the script folder.

Open config.cfg with a text editor and change the settings as needed. Set the input file to your newly downloaded XML file, choose the year you want to extract, set an output file, and set the file header, either <UNICODE-MAC> or <UNICODE-WIN>, depending on what platform you use InDesign on.

For now, leave all the style tags as they are so you can place the text into the example InDesign file and see how everything works. You can change them later and rerun the script

Finally, using the terminal or command prompt, navigate to the folder with the script and and run it by typing perl format_for_id.pl. If everything goes well you should have an output file at the location you specified, full of InDesign tags.

Open up Example.indd in InDesign CS3 or above and place the generated text file. All the text should come in perfectly with all the needed paragraph and character styles applied. Bravo!

Advanced usage

Obviously you’ll want to make some changes to the format of the output text. You might not want the post URL right after the tag—you might want it at the end, or not want it at all. With a little knowledge of Perl, you can edit the main script directly, mostly the combineSortClean() sub near the end of the script, to change the order of the output elements.

You can also disable tag indexing and allow the tags to be output with a paragraph style. Just comment and uncomment the appropriate sections in the code. The same goes for the author-specific character styles—comment and uncomment the needed lines in the script.

You can rename the styles and use your own—just make sure the styles exist in your InDesign document before you place the output file. InDesign will throw away any tags that don’t already exist in the document.

I made the script for our specific blog, so it doesn’t take every possible paragraph or character style into account. If you want additional functionality, you’ll have to add it. Feel free to fork the project off of GitHub and add to/improve it. That’s why it’s open source

If you have any questions, ask in the comments. Report any issues at the project GitHub page. I’ll try to respond quickly—I generally do, as evidenced by my pdftk-php project

Good luck!

Unfortunately, though, I wrote that tutorial as my first foray into the world of PHP/MySQL web development and had little idea of what I was really doing. Since then, however, I’ve done a fair amount of real-world web design and development, and even implemented this pdftk form system into a live, public-facing application.

In the interim, I’ve refined my system and released it as an open source PHP class, named pdftk-php. The project is hosted at GitHub, a brilliant hosting service for collaborative projects, which uses Git, the best version control software I’ve ever used. Anyone can check out, or clone, the project, make any edits to the core set of classes, and merge those with the main project branch—it is now a true community project. If you don’t want to contribute, you can still download it from GitHub as a .zip or .tar file.

Included in the project is a (hopefully) extensively documented example application that you can set up on your own server. You can try a live working example at pdftk-php.andrewheiss.com.

I’m working on writing a new step-by-step tutorial on how to set everything up, akin to the old one. In theory, the new pdftk-php should work with PDFs created in any program, not just LiveCycle. In the meantime, download pdftk-php, try it out, and report any bugs here on the blog or directly at GitHub (where I hope to keep everything related to pdftk-php from now on). Fork the project and contribute if you feel like it, too!

Thanks and good luck!

Read about my recent near misses at my post at our family blog, Heissatopia.