Archive for the ‘hackfests’ Category

End of March 2012 MapOSMatic hackfest

Sunday, April 1st, 2012

The hackfest is now finished! The last changes were committed by Thomas at 9:00 on ocitysmap module and at 9:30 on maposmatic module this April 1st.

MapOSMatic hackers at work. From left to right: Thomas, Étienne, Sylvain and Gaël

Overall we have committed 127 changes on the ocitysmap module (17848 lines added, 3713 lines removed) and 40 changes on maposmatic module (3532 lines added, 1544 lines removed) over the 7 days of the hackfest. Some of those changes were made at 2:36 in the morning.

Drawing, testing, designing, listing bugs, ...

Now, given a city name, we are able to automatically produce booklet maps, with a nice overview page, the map spread over several pages and an index referencing for each street or amenity the corresponding map page number and the square on this page. Each map page has pointers to map pages at the North, East, West and South of this map.

All those changes can be seen for France’s map on the development website (take care to empty your browser cache and reload the website from scratch).

Before putting them in production, we need to fix a few remaining issues (choice of scale on poster maps, a few typos here and there, take into account latest translations, …) and to wait for the end of the world OSM database import (started 7 days ago!).

And then you’ll can enjoy the new MapOSMatic! Have fun! :-)

(Part of) MapOSMatic team. Top: Frédéric, Gaël and Étienne. Bottom: David and Thomas

Posted in database, hackfests, maposmatic, ocitysmap, website | Comments Off

MapOSMatic hackfest, start of last day!

Saturday, March 31st, 2012

Once again, we have done significant progress yesterday.

Thomas worked on:

  • The front page for a multi-page map with a nice title, the map overview and the map information (last database update time, copyright, etc.);
  • A nice Donate page. We hope to be able to buy some hardware to increase the responsiveness of our server and to cover some of our expenses for such hackfest;
  • A small test suite. Not perfect but better than nothing. ;-)
  • The integration of all the work submitted by the three others of us.

Étienne worked on:

  • An overview map at the beginning of a multi-page map. It nicely shows all the individual pages on a small map of the city, with the corresponding page number for each one of them. Moreover, we now generates individual maps only for the area covering the chosen city, thus reducing the number of pages!
  • The index, taking care that listed streets are part of the chosen city and not adjacent ones;
  • Each map page in a multi-page map, putting a grey shade showing the limit of the chosen city.

Gaël, on his side, made several small but nonetheless difficult to achieve improvements:

  • He fixed a bug in the multi-page map index creation. All streets for letters considered the “same” in a given language (e.g. “e” and “é” in French) are now correctly put under the same index category;
  • He reduced the thickness of the grid on each individual map;
  • He added some missing street prefixes for French;
  • He fixed a lot of other small bugs;
  • He looked at the way we are projecting the map on a plane. In fact, in the future we might change the projection to avoid distortions on the map by leaving Mercator and choosing another projection (like the set of UTM ones).

On my side (David):

  • I improved the multi-page map street index so that it now wraps long names over several lines;
  • I re-implemented from scratch the B&W style sheet made by Sylvain in myself in order to have all of our changes at all zoom levels. There are still some bugs left, like remaining colours on tunnels.

The result of all this work can be seen on this new version of a multi-page map for Issy-les-Moulineaux. You can even test it on our development website (for France only, sorry).

Today, Frédéric is going to join us. We have to finish all the small details by the end of today!

Posted in hackfests, maposmatic, ocitysmap, website | Comments Off

MapOSMatic hackfest, start of day 7

Friday, March 30th, 2012

Yesterday, Thomas and myself made significant progress on the ulti-page rendering engine! We are now able to produce a PDF file of a city on several pages, with a small map overlap between each page. We also produce the global index at the end that references the correct square on each page. You can see the current work in progress on this map of Issy-les-Moulineaux (PDF, 9MB).

On his side, Gaël continued his previous work on improving the scaling code of ocitysmap, our back-end rendering code until a suitable patch could be sent to our mailing-list. He then started to work on the Javascript code used on the client side of MapOSMatic. Apparently there are lot of small stuff to fix and doing this is not that easy. :-)

On his side, Frédéric worked on the style-sheets, trying to understand how they are organized and making improvements on the installation documentation.

Today, Étienne will join us again. We have still a lot of work to do before the end of this week! Stay tuned… ;-)

Posted in hackfests, maposmatic, ocitysmap | Comments Off

MapOSMatic hackfest, start of day 6

Thursday, March 29th, 2012

Yesterday, Gilles Lamiral, author of imapsync, visited us. Yes the one behind the original MapOSMatic idea! We had a good time to discuss about MapOSMatic, OSM, Free Software, business models, pitfalls of modern Internet connections and much more.

Gaël has worked on scaling issues. At one point in the code we are rescaling the map to fit in a given paper size, with a given resolution. This part of the code triggers bugs, for example the badly placed one way arrows. Gaël has tried to improve that part of the code, removing unneeded scaling. Difficult stuff that is not finished yet but Gaël is making good progress.

Thomas and myself have worked on the multi-page renderer. Following on previous Étienne work, we are trying to add the ability to split a big PDF map into several pages with a common index at the end. Yesterday, we spent the whole day trying to split correctly the original map into smaller pages that overlap with a small margin. Gaël found our bug late in the night and now the computation seems correct.

BTW, a big thank to leaflet: it helped us produce some quick debug code to show the computed bounding boxes.

Posted in hackfests, maposmatic, ocitysmap | Comments Off

MapOSMatic hackfest, start of day 5

Wednesday, March 28th, 2012

On Monday and Tuesday, we have started to work on more difficult bugs and elaborated features.

Thomas has tried to find why the one-way street arrows on the MapQuest style sheet are wrongly placed. In fact, he found a bug in Mapnik2. But there are still some strange things occurring with this style sheet. :-)

Sylvain and myself (David), we have worked on a new “Black & White” style sheet. This style sheet is a derived version of original OSM Mapnik style sheet that contains less colours, for example on roads or buildings. This style sheet should produce much more readable maps when printing on black and white printers.

Étienne, Thomas and Gaël have started to work on an old feature request: maps cut into several pages, with a common street index. This feature is complex and difficult to implement, thus there is not much to show right now. Stay tuned! ;-)

Posted in hackfests, maposmatic, ocitysmap | Comments Off

MapOSMatic hackfest, day 3

Monday, March 26th, 2012

We are now starting the third day of the hackfest and some progresses have been made.

First of all, we have fixed small bugs that where hiding in the corners:

  • The display on thumbnails was breaking the website with Django 1.3, this is fixed;
  • The port number to access the GIS (Geographical Information System) is now a configuration parameter;
  • We use right and left arrows, displayed in bigger size, to navigate in the Create map dialogue;
  • We display a In progress icon when somebody is typing characters to look for a city name, giving a better feedback to the user on what the website is doing;
  • The user can now navigate with the Nominatim results, with Next and Prev buttons;
  • We display a message that explains why no Nominatim result has been found and how to fix it;
  • Hide the right arrow when the paper size is loading in the Create map dialogue;
  • When the same amenity appears several times in the same grid coordinate (e.g. several building of the same town hall), it appears only one in the index.

We have also started to work on small feature requests. Until now, we have implemented:

  • The display, on both the website and the generated maps, of the most recent date at which the OSM data has been updated;
  • The display of villages and hamlets in the index, a useful feature in rural areas.

You can see the all those features on the current development web site (for France only).

We are also working on the set-up of a new world OSM database in a suitable format for this new version of MapOSMatic. That way, we hope to deploy this new MapOSMatic to the world before the end of this week.

Posted in database, hackfests, maposmatic, ocitysmap, website | Comments Off

MapOSMatic hackfest in Rennes, starting up

Saturday, March 24th, 2012

Just a quick note to let our readers know that a MapOSMatic hackfest is taking place this week, from March, 24th to April, 1st in Rennes, France. David Mentré, Gaël Utard, Frédéric Lehobey as well as myself, who have already contributed to MapOSMatic, will be joined by a few other developers for a full week of intense coding around MapOSMatic.

We will try to publish blog posts throughout the week to let our readers know about what we are doing and where MapOSMatic is going.

Posted in hackfests, maposmatic | Comments Off

MapOSMatic hackfest in San Francisco: what happened

Saturday, March 24th, 2012

DotCloud

On Saturday, 18th and Sunday 19th of February, a two days MapOSMatic hackfest took place in San Francisco, in the offices kindly provided by DotCloud. David Decotigny, Maxime Petazzoni, Jérôme Petazzoni and myself, Thomas Petazzoni, were the participants of this short session of hacking around MapOSMatic. The result of those two days of work is not yet visible on the production web site, but we hope to make it available soon.

Amongst the things that have been worked on:

  • David has worked on understanding a Mapnik rendering bug that happens with “halo” text when the Cairo surface is scaled and rendered to a PDF file. This bug is currently very annoying for MapOSMatic because all text rendered with a “halo” in the background looks really ugly on PDF maps. We have reported this problem in August 2010 on Mapnik bugtracker (see this ticket, which has then be converted to Github to this ticket). After two days diving into the Mapnik source code, David found out that disabling the glyph cache used by Mapnik made the problem go away, but this workaround would presumably significantly affect the rendering performance, so it really is a workaround and not a solution. We hope to get some feedback from the Mapnik community about this, but since last month when David worked on the issue, we haven’t so far received any feedback.
  • Jérôme has worked on the monitoring of our servers. He has set up a monitoring website at http://stats.maposmatic.org/r2d2/. Amongst classical metrics (CPU, disk, etc.), Jérôme has added one special metric: the lag that our OSM database has compared to the official OSM database. Many of our users have often been asking why they are not quickly seeing their OSM changes in MapOSMatic maps. The diagram is visible at http://stats.maposmatic.org/r2d2/browse/gcc10.fsffrance.org/replication/ and shows this replication lag in seconds. At the moment, we are about 1.2 million seconds behind the official OSM database, which means about 14 days of lag.
  • Maxime has worked on the infrastructure as well, migrating the development version of the MapOSMatic installation to Mapnik 2, installing the MapQuest stylesheet, updating the OSM stylesheet, starting a new GIS database for importing the OSM data under the new schema expected by the latest OSM stylesheets.
  • On my side, I worked on fixing various minor issues on the web frontend that are currently preventing us from putting in production the current development version.
MapOSMatic Hackfest in DotCloud offices, San Francisco

MapOSMatic Hackfest in DotCloud offices, San Francisco. From left to right: David Decotigny, Jérôme Petazzoni and Maxime Petazzoni. I am behind the camera :-)

All in all, in just two days, we made a bit of progress, but not a lot. None of us had touched the project for quite a while, so it took some time to get up to speed, and two days were not sufficient to bring major results. However, we knew that a second hackfest would happen in France a month after, so the work will definitely continue !

Posted in hackfests, maposmatic | Comments Off

Hackfest MapOSMatic 2010, day 2

Thursday, August 5th, 2010
Maxime working on his TypeMatrix Dvorak keyboard, editing OCitySMap code with vim

Maxime working on his TypeMatrix Dvorak keyboard, editing OCitySMap code with vim

We’re early in the morning of day 3 of this MapOSMatic hackfest, everybody is still asleep, so it’s time to do a little wrap-up on what has been done yesterday :

  • Maxime and David have continued their work on refactoring the OCitySMap module and extending it to support the new level of map customization that we will offer. So far, the map generation is working again, together with the overlayed grid, with a much, much better Python code, that will hopefully allow us and other contributors, to add more features more easily than in the past. Today, they will continue their refactoring work with the street index generation.
  • Gaël has continued and completed his work on the slippy map user experience improvements. The user now draws a rectangle to define the area to be rendered, and we immediately check whether the area is too big to be rendered by MapOSMatic.
  • Éric has continued his experimentation of the ReportLab PDF library and its Platypus page layout engine, but finally came to the conclusion that while the automatic page layout engine is nice, it doesn’t provide enough control on the layout to generate a street index similar to the one we’re doing today.
  • Pierre has worked on different small but important improvements:
    • in the page corresponding to a particular rendering, add a link to the corresponding location in OpenStreetMap so that an user can easily check the current status of a particular area
    • in the results provided by our Nominatim proxy, add information about the size of each administrative boundary. This will allow us to no longer limit the selection of administrative boundaries to level 8 boundaries while still preventing the user from rendering too large areas.
  • I’ve worked on finishing the Map Creation Wizard that allows to select the different parameters for a map. It is now feature-complete, the code has been cleaned up, even though it still need some more polishing, especially on the presentation side, which will done later by Maxime, our CSS expert.

Last, but not least, the import of the planet database has completed on our new server that should start hosting the official MapOSMatic.org website in a couple of days or weeks. With a better CPU, two times the amount of RAM and four times the amount of disk of the previous server, it should provide a more scalable hosting solution for the coming months.

Today, work will continue with a first integration of the different improvements, the rest of the OCitySMap refactoring, experiments on Cairo for the booklet generation, improvements on the street index generation, experiments on rendering Wikipedia pages, final details on the wizard, and more.

Stay tuned for tomorrow updates !

Posted in hackfests | Comments Off

Hackfest MapOSMatic 2010, day 1

Wednesday, August 4th, 2010
Notes on new MapOSMatic features

Notes on new MapOSMatic features

As we announced during our talk at State Of The Map, we are currently having a hackfest during which we’ll try to improve MapOSMatic in various directions until Sunday. After a startup phase yesterday evening with the arrival of all participants and the preparation of working setups of MapOSMatic/OCitySMap, the hard work has started today.

We’ve spent the whole morning discussing the design of the improvements to MapOSMatic we’d like to implement during this hackfest. The following topics have been discussed :

  • Better adaptation of MapOSMatic results to printed maps, in terms of format. To solve this, what we intend to do is that once the user has selected a geographic area (either from bounding box or through the name of an administrative boundary), we’ll offer a selection of suitable paper sizes, from a list of standard formats. Obviously, the range of paper formats available for a given area will be limited by the size of the area: the bigger the area is, the larger the paper size will have to be. Once we know the paper format choosen by the user, we will adjust the geographic area to fill the complete paper area, in a way that of course guarantees that the complete originally selected area is visible. If a lot of paper space is available compared to the size of the geographic area, we might even increase the zoom level to provide additional map details.
  • Customization of the rendering layout. Instead of our current single layout with the map on one side and the street index on the other side, with the same size as the map, we will offer three rendering layouts : a plain layout (just the map, no index), a map with an index (the index being on the same side as the map) and a booklet layout (where the map and the index is split in several pages, for easy printing on A5/A4/US letter paper formats)
  • Customization of the stylesheets. To begin with this, we will just provide a choice between a pre-selected set of Mapnik stylesheets. Offering the ability to use a custom Mapnik stylesheet is relatively complex in terms of security issues (Mapnik stylesheets contain SQL requests), so we decided to keep the choice inside a limited list. Of course, we’d like to see people contributing additional Mapnik stylesheets.

After the design discussion, we’ve written a few Python lines of code to define the new interface between MapOSMatic (the web frontend) and OCitySMap (the rendering backend), so that different sub-teams could work separately on these modules. And then, the implementation work started, in different areas :

  • Major refactoring of OCitySMap, to prepare the addition of the new features. The OCitySMap code, while relatively short, was pretty ugly in various areas, so a refactoring was more than needed. Hopefully, it should allow us in the next few days to add features without too much pain :-)
  • Work on the OpenLayers slippy map to improve the user experience for the selection of the geographic area. Now, the full area visible on the slippy map will not be area selected by rendering. Instead, the user has to draw a rectangle of the area to render, and this rectangle remains visible on the map so that the user can adjust the area as needed.
  • Work on the creation map process in the MapOSMatic web front-end by switching to a wizard-like procedure. This is needed since we will now have additional steps to configure the different aspects of the map before rendering it. This involves quite a bit of Javascript, JQuery, CSS and OpenLayers…
  • Experimentation on the ReportLab PDF rendering library to generate the index, as it would probably be much easier to use than our manual index rendering code directly using Pango and Cairo.

More about our work tomorrow !

Posted in hackfests | Comments Off

Major improvements to MapOSMatic

Monday, January 4th, 2010

We have just sent the following announcement to OpenStreetMap lists, officially announcing the improvements made to MapOSMatic during the latest hackfest.

As a new year’s present, the MapOSMatic team is proud to announce that a new version of the maposmatic.org website has been put online, with major improvements over the initial version announced in September 2009.

For the record, MapOSMatic is a website that allows to generate city maps from OpenStreetMap data. Each map is divided into squares to easily find streets and is delivered with the corresponding street index.

The new MapOSMatic provides the following improvements :

  • Support for the whole world. Any location in the world can now be rendered on maposmatic.org.
  • OpenStreetMap database updated daily. Until now, the database had never been updated since the service was started in September 2009. Now, the geographic database used to render the maps is updated daily, providing maps with the latest contributions to OpenStreetMap. Each map contains the date at which it was generated.
  • Better city search engine. Thanks to Nominatim, we now provide a search engine that allows to find cities in a much more usable way: cities with the same name can be distinguished and the search works even when the city name is not completely correct.
  • Support for other languages. A few parts of the map rendering process is language-dependent and we now have the infrastructure to use language-dependent code. For the moment, we support English, French and Italian, but we are waiting for your contributions to support other languages. The website has also been translated to German and Italian.
  • Amenities in the index. In addition to the streets, we have added important amenities to the index: schools, town hall, post offices, places of worship, etc.

All these improvements are available now on http://www.maposmatic.org

You can follow the progress and improvements of MapOSMatic on our blog at http://news.maposmatic.org. MapOSMatic is of course free software, you can fetch its source code and contribute to the project, see http://savannah.nongnu.org/projects/maposmatic/.

Do not hesitate to send us your feedback, comments, suggestions and contributions to [email protected]

Cheers,

The MapOSMatic team.

Posted in hackfests, maposmatic | No Comments »

MapOSMatic hackfest, day 3

Tuesday, December 22nd, 2009

Yesterday was day 3 of the hackfest, and we of course spent our day working on MapOSMatic.

  • Support for amenities in index has been added. Now, in addition to the streets, you have the location of the town hall, police station and other public buildings, the schools, universities and other education buildings, and the places of worship. More improvements are needed in this area (translation of headings, support for amenities represented by polygons, fix incorrect grouping), but the general infrastructure is in place.
  • The main form to select the city has been further improved, with bug fixes, improved usability, etc. Said like this, it doesn’t sound like a big deal, but all this stuff is written in Javascript using the JQuery library and requires careful fine-tuning to be working. We easily spend a day on the same 20 lines of Javascript.
  • The language selection has been improved, with all french-speaking countries and all english-speaking countries added (we’re waiting for your contributions for other languages). The list of languages available is now automatically reduced to the languages spoken in the country in which the selected city is. For multi-lingual countries like Belgium, we have no way to know if the selected city is in the french-speaking area, the dutch-speaking area or the german-speaking area. Therefore, it is up to the user to choose between these languages. This selection changes how the streets are sorted and the list of prefixes that we consider before sorting.
  • The slippy map based on OpenLayers is now loaded only if bounding box mode is used. This allows a much quicker loading of the MapOSMatic homepage.
  • We have improved our server infrastructure to regularly update the coast lines, which are coming from a difference source than the planet OSM xml file. It allowed to fix rendering problems in island where we had ugly coast lines. These informations are now updated once a week.

We’re now starting day 4 of the hackfest. We will soon send out a notification to beta-testers so that they can test all these new features and report their comments and bugs.

Posted in hackfests, maposmatic | No Comments »

MapOSMatic hackfest, day 2

Monday, December 21st, 2009

Yesterday was day 2 of the hackfest, and we did a number of improvements :

  • Cities can now be searched in an efficient way, thanks to the usage of Nominatim. The “Administrative city” field has been turned into a small search engine that returns, through Javascript, the list of cities matching the request. We spent quite some time understanding the results of Nomimatim and how they relate to the objects in the PostGIS database generated by osm2pgsql. We still need to fix a few issues with this Javascript-based search engine.
  • We used to identify cities by their name, but this of course doesn’t work since many cities have the same name around the world. So, now, rendering of city by administrative boundary are identified by the OSM id of the polygon in the PostGIS database. OCitySMap has been modified consequently, and jobs submitted through MapOSMatic are properly handled by maposmaticd.
  • The language used to generate a map can now be selected before generating the map, and several other improvements have been made on the internationalization. The main topic remaining is to pre-select the languages that are useful depending on the country in which the city being rendered is in.
  • The slippy map integration has been reworked to post-pone the loading of the tiles so that the homepage of MapOSMatic is much faster to load.
  • The documentation to install MapOSMatic and OCitySMap has been further improved.
  • Several bugs or tasks waiting in our bug tracker have been fixed, and some cleanup has been done on bugs that were already fixed by previous development

We expect to put the new version into production tomorrow, with the main achievement of having world-wide support, a better city search feature, and internalization for maps. Of course, we’ll need a lot of contributions for the internationalization support.

Posted in hackfests, maposmatic | No Comments »

MapOSMatic hackfest, day 1

Sunday, December 20th, 2009

The hackfest started just yesterday around noon with the arrival of six crazy hackers who will spend four days working on MapOSMatic. For the first day, we spent some time setting up MapOSMatic on the laptop of all developers and then started working on different topics :

  • A group started experimenting with Nominatim in order to improve the city lookup feature of the MapOSMatic website. This is very important for several reasons. First because for the moment our city lookup mechanism only finds the city if the entered name matches exactly the one in the database (spaces, dashes and accents included). And also because we have no way of letting the user choose between several cities of the same name, which is very important as we intend to put the worldwide database into production soon. The initial work we did is to add a small gateway to Nominatim in MapOSMatic (see this commit). The rest of the work is being worked on now.
  • Another developer worked on the infrastructure to allow the adaptation of OCitySMap to other languages: how the street names should be sorted, what are the common prefixes for streets, places, roads, etc. See this commit, this commit, this commit, this commit, this commit and this commit. Support is only for fr_FR and en_GB at the moment, and only at the OCitySMap level. Nothing has (yet) been implemented to select this from the MapOSMatic website, but it’s obviously on the TODO-list for this hackfest.
  • The documentation for installing OCitySMap and MapOSMatic has been improved and updated for newer distributions.
  • The german translation contributed by malenki has been added to the website and now allows to access the MapOSMatic website in German. Contributions for other languages welcome, contact us for details.

The previous hackfest took place in August in Toulouse (south of France) where we enjoyed a very nice and warm whether, wearing shorts, eating tomatoes and cucumbers straight from the garden. It was just a perfect summer hackfest. Now, we’re in Méridon (near Paris), where snow has fallen since Friday and was falling again today, so we are enjoying a perfect winter hackfest, with a nice tartiflette yesterday evening ! :-)

Posted in hackfests, maposmatic | No Comments »

MapOSMatic hackfest next week

Tuesday, December 15th, 2009

The MapOSMatic application website was initially created in August 2009 by a group of free software enthousiasts during a one-week intensive hackfest. The same group of people is going to meet again for a 5 days hackfest next week and we plan to take advantage of this hackfest to work on the major issues of MapOSMatic :

  • Put the world, daily-updated, OpenStreetMap database into production, so that cities from all over the world can be rendered and up-to-date data is used
  • Improve the city selection, to make it fuzzy, and to support various countries
  • Improve the city index generation to support different languages
  • Improve the web site display of existing renderings

Of course, a lot more issues are registered in our task list and in our bug tracker, but we will very likely focus on the points mentioned above.

Posted in hackfests, maposmatic | No Comments »