Dispatches from somewhere far away

Stuff to learn in 2009

January 6th, 2009 Chris

Just going to write all this down so I don’t forget. Could call it “resolutions” but that whole mindset seems designed to produce regret come December. Let’s just make this a to-learn/to-do list:

GeoDjango

With Tools for News up and running and RedFence 2.0 almost there, I feel like I’ve got a good enough grasp of Django to try out the GIS branch. I’ve got a subdomain for it set up and a couple project ideas to play with. No promises that anything interesting will happen there, but it’s there.

JavaScript

I have a well-served bias against everyone’s favorite client-side scripting language, mostly due to the number of websites I’ve seen broken by it. But I do need it to do proper mapping and work with a lot of APIs. Big question: Go straight to jQuery or get a solid grounding in basic JS first. Might depend on which tutorials I find. Thoughts?

Travel more

OK, not really something to learn, but definitely something to do. As of February, I’ll have been back in the States a year. Time to get moving.

More ideas

  • Screen scraping
  • Rebuild this blog and my portfolio in Django
  • Apply for a Knight Challenge Grant

What else should I be learning?

Blueprint CSS. Yes.

December 31st, 2008 Chris

Why, or why, did it take me so long to discover Blueprint CSS?

Because I wasn’t looking, clearly.

I am not a designer. Designers are people with style, and my sister, my girlfriend and my housemate have all made it very clear that I’m lacking in that department. More than that, CSS fits into the large category of things I’d much rather outsource to a competent professional. I’ll stick to Python and prose.

Clever readers of yesterday’s post announcing the beta launch of Tools for News may have noticed the line about building the core of the site in about two days. Two days in late October. And such clever readers may have asked themselves, “What happened over those two months?”

Not much. Mostly, because the site was naked, and I didn’t want to send it out into the cold that way.*

For much of its life in development, the site had no style whatsoever. I built it in plain HTML and spent most of my time working on the back end (models and views) and making the template and a few custom tags work.

Then I found Blueprint CSS.

Blueprint is a CSS framework that, like Django does for Python, abstracts common tasks and give designers a head start. For me, it meant more than skipping past the tedium of starting a CSS file from scratch. Blueprint turned out to be all the style I needed to give the site a clean, grid-based layout and clear, readable type.

And skipping past the tedium (for me) of CSS means I can focus on getting stuff built. I’d call that a win.

*There was also much work to be done on another project, a magazine CMS, which I’ll blog about soon. You can see what projects I’m working on and follow their progress here.

New tools for new news

December 30th, 2008 Chris

Journalists need new tools to work online. In the last year, I’ve used more that I can count, most of them free, to find and tell better stories on the Web.

Back in October, I started building an online database of such tools as a personal project, just a way to keep track of everything I was using. It has since grown into something I think others will find useful, so I’m releasing it into the wild.

Tools for News

The site is in public beta for now. Eventually, I hope to move it to its own domain.

Anyone can browse this site and subscribe to an RSS feed of the latest tools. Registering allows you to add new tools, add links to existing tools and bookmark tools, which will be saved on your contributor page.

Try it out. Find bugs. Send feedback.

Production notes:

The core of the site was built in about two days using Django. I started building it after seeing other bloggers posting lists of tools they use, or that others should use. I wanted a better way to find and organize all of this.

I leaned heavily on reusable Django apps, taking to heart what James Bennett often says: The ability to create focused, reusable applications is Django’s true killer feature. It is. I used three of his apps for this site and four on another, not counting pieces of code inspired by what Bennett and others have given away.

For inspiration, I looked at Django Snippets (built by Bennett, who used it as an example project in his excellent book) and Django Friendly, a site that rates hosting providers for, well, Django friendliness.

Credit also goes to Ryan Sholin, Zac Echola and Rick Martin letting me bounce ideas off them. Thanks guys.

Local is what local covers

December 23rd, 2008 Chris

Following up on my last post, I started listing in my head all the places and non-places my local newspaper, like every paper I’ve read or worked for, covers. Here’s a partial list for a few news organizations:

The Press Democrat (Santa Rosa, CA):

Places:

  • Santa Rosa
  • Rohnert Park
  • Petaluma (bureau)
  • Sebastopol
  • Winsdor
  • Healdsburg
  • Graton
  • Guerneville
  • Ukiah (bureau)
  • Lake County
  • Mendocino County
  • Occasional ventures into Napa, Marin and San Francisco

Non-places:

  • Sonoma State University (just outside Rohnert Park)
  • Santa Rosa Junior College (in Santa Rosa, Rohnert Park and Petaluma)

The Fremont Argus

Places:

  • Fremont
  • Newark
  • Union City
  • Niles Canyon

The Hayward Daily Review

Places:

  • Hayward
  • San Leandro
  • San Lorenzo
  • Castro Valley

Non-places:

  • CalState East Bay
  • Chabot College
  • Hayward Executive Airport

The Signal (Santa Clarita, CA):

I grew up here, reading this paper

Places:

  • Valencia
  • Saugus
  • Canyon Country
  • Newhall
  • Castaic
  • Agua Dulce
  • Placerita Canyon

Non-places:

  • College of the Canyons
  • Wm S. Hart School District
  • CalArts
  • The Masters College
  • Henry Mayo Newhall Memorial Hospital

None of these papers are major metros. The Press Democrat, by far the largest, peaked at around 92,000 daily subscribers a few years ago. The rest are small to mid-sized local dailies, covering cities between big cities with small (and shrinking) staffs.

Not all these places I’ve listed need a dedicated beat reporter, but online, it’s not hard to turn any definable place into a focused page with a permanent URL and an RSS feed. A simple tagging structure–ideally separate from topical tags–would get us our local news feed, without any users having to hack something together on their end, and nothing harder for reporters than writing a dateline. It would make me happy.

Finding a local news feed

December 19th, 2008 Chris

My next little side project is going to involve parsing feeds. I’m tired of wading through hideous newspaper.coms trying to find a certain story, or stories about a certain area, without having to avoid national news I’ve read elsewhere, or bits about towns I’ll never visit.

Andrew Meyer has been having the same problem:

When I visit PressDemocrat.com, I go for one thing: Sonoma County news. Someone in Mendocino County might visit the site for Mendo County news, which is great, but not the reason I visit. Ok, with that said, how do I locate Sonoma County news on PressDemocrat.com. Ahh… herein lies the problem. Local news granularity is sorely missing on the site.

When scrolling down PressDemocrat.com’s frontpage, you won’t find sections for “Santa Rosa news” or “Windsor news”

I’m in the same boat. I live in downtown Santa Rosa, and Windsor, sorry folks, isn’t at the top of my reading list (I drove through it once, nice looking place). Nor is Mendocino County, or Petaluma, or Napa, really. And chances are, people in Mendocino and Petaluma and Napa and Windsor care a lot more about what is going on in those places–and just those places–than they do about what happens in my front yard.

(I should note here that I don’t actively ignore places I don’t live, but I don’t follow what happens two towns over with the zeal that I watch what happens close to home, or in China or Washington, DC, for that matter. This isn’t an argument in favor of insularity–I won’t make that one–but simply a need for better filters.)

So, assuming the news organizations of the Bay Area don’t decide to rebuild their sites with granular news feeds (like Spokane just did), how can I get a feed of just a few places that I really want to follow? Looks like I’m going to have to build my own feed.

How do we do that?

Start with a few basic ingredients:

  1. A news-type site with an RSS feed
  2. Yahoo Pipes
  3. Patience
  4. A feed reader (I like Google Reader)

Here’s a simple result:

I picked up the Press Democrat’s local news feed, then fed the 50 most recent items through a Pipes filter, allowing only stories that contained “Santa Rosa” in the headline or lede paragraph. We end up with four stories. Not quite what I’d hoped.

I know this misses a lot. It missed, for instance, two stories about Mark Felt, better known as Deep Throat. He died yesterday in Santa Rosa.

Part of the trouble is the PD’s feed: It only gives out an excerpt of each story, usually the first sentence or two. To my knowledge, only the Guardian (UK) offers a full RSS feed.

What I’d really like to do (assuming, of course, that newspapers around the Bay don’t decide to be more like the Guardian or the Spokesman-Review) is follow each link, look at the text, and grab those that have Santa Rosa (or any location) anywhere in the main body of the story. Something like Open Calais would be sweet.

Anybody know how to do that?

DalianDalian was down

November 23rd, 2008 Chris

UPDATE: It’s back online.

Apologies to anyone looking for bars, restaurants or news about Dalian. The site is down with a bad case of DOS.

We (by which I mostly mean Alex) are looking into the matter, which may or may not involve new hosting.

Geotagging: News here, now (or there, then)

October 31st, 2008 Chris

In a comment on yesterday’s post about making news easy to find and easy to share, Alex reminds me that he and I have had this conversation before. Now that I think about it, we’ve had this conversation a lot, especially about finding relevant news based on location.

And about this time last year you and I worked out a way to do this. Well, the skeleton of one. It can be built, but UI is important, and location’s not going to happen unless Geotagging is made easy, and no one’s going to use it unless everyone else is using it.

There must be a better way than to get someone to find a zip/postal code or click around a map.

A fundamental problem:: there’s a lot of content in content silos. And aggregation of the full text is too much often a legal issue than a technical one. Put NYT, WSJ, Times, Guardian, Xinhua, Al Jazeera content into a giant silo and get the parties to agree, include the blogosphere, Wikipedia, geo tag and auto-classify the lot. I believe that would be hard.

Geotagging was always going to be the hardest thing. I know a couple news sites that do it, but it’s rare.

  • Everyblock links to stories that have enough geographic information to plot on a map.
  • MySanAntonio.com uses MetaCarta to create a local news feed (but good luck finding that on the site).
  • Pegasus News remembers your neighborhood (if you’re logged in) and filters news, events and listings accordingly.

We tried doing this with DalianDalian. It works well enough for restaurant and bar listings (when I log in, the map centers on my old address on Xinhua Jie) but we never figured out how to make the process simple enough for other types of content, nor could we automate geotagging for imported blog posts and photos.

Of all the sites I’ve seen, Everyblock has by far the best geotagging system. It’s automated, uses natural language and has the cleanest user interface. The source code is set to be released in June 2009, so it’s possible some of that functionality can be integrated into sites producing news, not just aggregators.

But it doesn’t even need to be that complex. I can live without news filtered by city block. Just city-specific news would go a long way. InsideBayArea.com does this (here’s Fremont) in the most basic way (disclosure: I freelance for this company).

Again, all of this goes back to what I wrote before: When I go to a site, all I want is news, findable and shareable, without having to swim through clutter. And news about somewhere I’m not (on a local news site) becomes clutter very quickly.

Easy to find. Easy to share.

October 30th, 2008 Chris

I come back to this thought again and again in my head:

I don’t need more video, or more multimedia of any kind, or even databases or forums or yet another social network. All I want, as a reader, is news that is easy to find and easy to share. It’s what I want in the sites I build and the newsrooms I work for, too.

Is this too much to ask?

Most days, I practically live in Google Reader. I have become so addicted to RSS, to the plain but well-structured layout of the feed, that anything that comes between me and the content feels like clutter, an obstacle to be pushed aside, or an annoyance that makes me not want to come back. I am spoiled, but so is anyone who gets their news primarily off Google’s or Yahoo’s pages. All I want is content.

Here’s how I’d like to find news, based on my own priorities:

  • By location: What’s close to home? If something–crime or safety related, especially–happens on my street, I’d love to know about it.
  • By date: What’s recent? What happened last month, last year, a decade ago?
  • By popularity: What are people talking about? What’s holding people’s attention and getting more than drive-by hits?
  • By topic: What is this story about? When something keeps coming up in the news, I’d love to see everything written about that on one page (like the Times Topics. Can’t we all do this?).
  • By author: Sometimes, I know who I like to read, and not just because a lot of my friends are journalists. This isn’t hard with a good database back end (again, the Times does it).

When the San Jose Mercury News asked readers how they find news (in general, not just what’s in the newspaper), their top two answers were word of mouth and Google. Online this means we read what others send us, or we go looking for it. The questions for news organizations, then: How do we make that process easier, and how do we make sure our content gets to those who want it?

Whose job is it?

  • Designers: Clean UI is a godsend. Just give me something that doesn’t make my eyeballs bleed.
  • Developers: Can a brother get a human-readable URL? How many clicks must I go through to find what I’m looking for? How many ways can I find a story?
  • Editorial: SEO those headlines. Think about how readers use the site and find your content. Better yet, ask.

More importantly, take the website seriously. Give it the respect and attention the print product gets. I wish I didn’t have to say this in the last quarter of 2008, but here we are.

A word for managers

October 28th, 2008 Chris

From Joe Grimm, until recently of the Detroit Free Press, in his most recent (and always-excellent) Ask the Recruiter column:

We imperil our companies and our own careers when we do not listen closely to young people, whose experience with media is so different than our own and whose ideas may hold some of the solutions. Invite them into the strategy sessions, encourage them to really brainstorm and explain — and then try some new ways.

Joe also has a blog.

A short history of DalianDalian.com, and why you should build your own

September 17th, 2008 Chris

When I arrived in Dalian, most of what I knew about the city came from word of mouth. I’d spent a few months hanging around a local expat forum, reading blogs, emailing people who lived there.

I read up on the city where I could, but coverage of smaller cities in China (even small cities of three to six million) tends take a birds-eye view. I knew about Thomas Friedman’s ongoing love affair with the Northeast’s biggest outsourcing hub, and I knew about the Sino- and Russo-Japanese wars. There was more out there, but it was scattered among blogs and forum posts. There were bits on other websites and far more in people’s heads.

When Alex, Rick and I sat down to build DalianDalian, we wanted to pull all those threads together. Our site would be a hub, pulling in and linking out to every piece of information we could find about Dalian and Northeast China.

A few days ago, Thomas Crampton interviewed Alex about the site for his blog and Danwei.org:

In comments, Alex added more on the what went into building the site:

* Web development has got to the stage where someone with little technical knowledge can create a multi-faceted site.

* Know what you really want to do before starting to do it - that’s really really important in any project, large or small.

* Choose a business model. Do you want to be able to earn an income from the site? In a 2nd or 3rd tier Chinese city this will be hard, almost futile, and will run into many problems related to regulation, receipt issuance, etc. It is possible to do these things on the side, but I wouldn’t recommend it.

* Know the tools available. We started with Drupal because it was really the only (free) thing at the time capable, but needed quite a lot of tweaking, especially modules that were half-finished. Now there are good alternatives, probably Ning is the most well known, but make sure they can do what you really want in the way you want (i.e. have some specifications, as mentioned above). Doing it again I’d still go with Drupal, the additional modules are now much more mature and while it is pretty administer friendly it offers lots of opportunity to go beneath-the-hood if desired.

* Ask someone for help. Drupal is a good community full of free software volunteers happy to give advice. They’re not going to build a site for you for free though they will make it easier. I’m happy to give some advice and technical tweaks for something similar in nature to DalianDalian.com.

* Stay on the move and be self-critical. What do the members and other contributors say? What don’t you like? This is very much our problem (we like to break the site to see if a touch-up is possible, it is a work in progress).

In the end, making a website, creating media, is fascinating, at least we think so. If you’re in a 2nd/3rd tier city in China or anywhere in the world that you think under-serves a niche, a little tinkering with technology can get it served; it’s far less ‘making a website’ far more thinking deeply about what is useful and ways to be useful.

Alex’s last point is worth a post in itself. We learned far more about media and community building the site than we did about web development (of which we learned a great deal). Anyone thinking of starting a similar project, do it. If you need help or advice, get in touch.