Archive for the ‘maposmatic’ Category

« Older Entries

New storage for MapOSMatic.org

Tuesday, May 27th, 2014

You probably noticed that our copy of OpenStreetMap database has not been updated for more than two months. In fact, our current storage space of about 500 GB was completely filled by the database. :-(

The good news is that we are currently in the process of changing that storage! Two new Samsung S840 Evo 1 TB disks have been ordered, that will quadruple our storage capacity to 2 TB. :-)

Once the disks are installed, we will need to make a new fresh import of OpenStreetMap database. We hope to do this import in hstore format that would allow us to bring new features to MapOSMatic.

Stay tuned!

Posted in database, maintenance, maposmatic, website | Comments Off

MapOSMatic is back!

Friday, November 30th, 2012

Good news everyone: MapOSMatic is back!

Several months ago, our copy of the OSM database started to have major issues and got corrupted, unfortunately beyond recovery. We needed to re-import the full database. At the same time, we were having performance issues with the OSM data replication because the traditional, rotational the hard drive of the database server was not fast enough and we couldn’t keep up with the rate of updates to the OSM database. And last but not least, OSM was changing its license which had a strong impact on the data in our database too of course!

Therefore “The Plan” was to move our database server to a new machine that had SSD drives and import on this fast database server a fresh copy of the OSM database. As you probably discovered, it took us much more time than we originally planned ;-)

Today, we are very pleased to announce that the new database server is now up and running and it is fast! MapOSMatic is back on and once again available to the OSM community! :) And a big thank you to the FSF France for giving us access to a powerful server to host our OSM database.

Please enjoy our service! We have a lot of queued jobs since the summer that the rendering pipeline is now processing. It will take a few days before all of them are processed, but rest assured the server is working hard to catch up :) We will make sure to send a note when the queue is empty and new renderings can be processed as soon as they are submitted.

- The whole MapOSMatic team (with a special thanks to Maxime that does all the admin stuff!)

Update Dec 5th, 2012: we have processed all the pending renderings in the queue since July. You can now submit new map rendering jobs and they will be processed right away!

Posted in database, maintenance, maposmatic, website | Comments Off

Service interruption

Tuesday, September 18th, 2012

Our OSM database is currently down therefore the MapOSMatic website no longer works. We currently don’t know how long it will take to restore the system.

Edit 2012-09-18 16:24: Our database is corrupted and PostgreSQL does not restart. It might take a long time to restore a new one.

Posted in database, maposmatic, website | Comments Off

New version of MapOSMatic available!

Thursday, April 19th, 2012

In September 2009, we launched MapOSMatic, a free web service that allows to render city maps on-demand based on OpenStreetMap data. Those city maps, divided into squares, are
associated with a street index, making the process of locating a street on the map easier.

Multi-page map

We are proud to announce today the launch of a new version of MapOSMatic, which is the result of significant development efforts. Amongst the new features:

  • The rendering of poster maps is now done on large standard paper formats (A3, A2, A1, etc.), automatically chosen depending on the geographical size of the city, instead of arbitrarily-sized papers that were hard to print. The end result is close to commercial folded maps;
  • The availability of several rendering styles. For now, we provide the standard OpenStreetMap.org style, several styles provided by MapQuest, and a custom style more suitable for printing. In the future, we expect to extend those styles, or even to let users provide their own styles. Do not hesitate to contact us about custom styles;
  • Improvements in the selection of cities: in the previous versions, we were limited to OpenStreetMap areas of a certain administrative level;
  • And many, many other smaller features and improvements: quality of the renderings, better user interface to render maps, last OSM database update date printed on the map, etc.

MapOSMatic is completely free software, distributed under the terms of the Affero General Public License v3. The project is available through Git repositories, has a mailing-list and an IRC channel. For details, see our About page, our wiki and the Savannah project page.

In addition to the launch of this new version, we are also starting a donation campaign. The project is completely developed and maintained by volunteers, but we need funding to cover hardware costs and transportations costs to organize the developer meetings during which MapOSMatic improvements are implemented (see this blog). If you appreciate MapOSMatic, do not hesitate to help us by donating with PayPal.

Posted in maposmatic, website | Comments Off

End of March 2012 MapOSMatic hackfest

Sunday, April 1st, 2012

The hackfest is now finished! The last changes were committed by Thomas at 9:00 on ocitysmap module and at 9:30 on maposmatic module this April 1st.

MapOSMatic hackers at work. From left to right: Thomas, Étienne, Sylvain and Gaël

Overall we have committed 127 changes on the ocitysmap module (17848 lines added, 3713 lines removed) and 40 changes on maposmatic module (3532 lines added, 1544 lines removed) over the 7 days of the hackfest. Some of those changes were made at 2:36 in the morning.

Drawing, testing, designing, listing bugs, ...

Now, given a city name, we are able to automatically produce booklet maps, with a nice overview page, the map spread over several pages and an index referencing for each street or amenity the corresponding map page number and the square on this page. Each map page has pointers to map pages at the North, East, West and South of this map.

All those changes can be seen for France’s map on the development website (take care to empty your browser cache and reload the website from scratch).

Before putting them in production, we need to fix a few remaining issues (choice of scale on poster maps, a few typos here and there, take into account latest translations, …) and to wait for the end of the world OSM database import (started 7 days ago!).

And then you’ll can enjoy the new MapOSMatic! Have fun! :-)

(Part of) MapOSMatic team. Top: Frédéric, Gaël and Étienne. Bottom: David and Thomas

Posted in database, hackfests, maposmatic, ocitysmap, website | Comments Off

MapOSMatic hackfest, start of last day!

Saturday, March 31st, 2012

Once again, we have done significant progress yesterday.

Thomas worked on:

  • The front page for a multi-page map with a nice title, the map overview and the map information (last database update time, copyright, etc.);
  • A nice Donate page. We hope to be able to buy some hardware to increase the responsiveness of our server and to cover some of our expenses for such hackfest;
  • A small test suite. Not perfect but better than nothing. ;-)
  • The integration of all the work submitted by the three others of us.

Étienne worked on:

  • An overview map at the beginning of a multi-page map. It nicely shows all the individual pages on a small map of the city, with the corresponding page number for each one of them. Moreover, we now generates individual maps only for the area covering the chosen city, thus reducing the number of pages!
  • The index, taking care that listed streets are part of the chosen city and not adjacent ones;
  • Each map page in a multi-page map, putting a grey shade showing the limit of the chosen city.

Gaël, on his side, made several small but nonetheless difficult to achieve improvements:

  • He fixed a bug in the multi-page map index creation. All streets for letters considered the “same” in a given language (e.g. “e” and “é” in French) are now correctly put under the same index category;
  • He reduced the thickness of the grid on each individual map;
  • He added some missing street prefixes for French;
  • He fixed a lot of other small bugs;
  • He looked at the way we are projecting the map on a plane. In fact, in the future we might change the projection to avoid distortions on the map by leaving Mercator and choosing another projection (like the set of UTM ones).

On my side (David):

  • I improved the multi-page map street index so that it now wraps long names over several lines;
  • I re-implemented from scratch the B&W style sheet made by Sylvain in myself in order to have all of our changes at all zoom levels. There are still some bugs left, like remaining colours on tunnels.

The result of all this work can be seen on this new version of a multi-page map for Issy-les-Moulineaux. You can even test it on our development website (for France only, sorry).

Today, Frédéric is going to join us. We have to finish all the small details by the end of today!

Posted in hackfests, maposmatic, ocitysmap, website | Comments Off

MapOSMatic hackfest, start of day 7

Friday, March 30th, 2012

Yesterday, Thomas and myself made significant progress on the ulti-page rendering engine! We are now able to produce a PDF file of a city on several pages, with a small map overlap between each page. We also produce the global index at the end that references the correct square on each page. You can see the current work in progress on this map of Issy-les-Moulineaux (PDF, 9MB).

On his side, Gaël continued his previous work on improving the scaling code of ocitysmap, our back-end rendering code until a suitable patch could be sent to our mailing-list. He then started to work on the Javascript code used on the client side of MapOSMatic. Apparently there are lot of small stuff to fix and doing this is not that easy. :-)

On his side, Frédéric worked on the style-sheets, trying to understand how they are organized and making improvements on the installation documentation.

Today, Étienne will join us again. We have still a lot of work to do before the end of this week! Stay tuned… ;-)

Posted in hackfests, maposmatic, ocitysmap | Comments Off

MapOSMatic hackfest, start of day 6

Thursday, March 29th, 2012

Yesterday, Gilles Lamiral, author of imapsync, visited us. Yes the one behind the original MapOSMatic idea! We had a good time to discuss about MapOSMatic, OSM, Free Software, business models, pitfalls of modern Internet connections and much more.

Gaël has worked on scaling issues. At one point in the code we are rescaling the map to fit in a given paper size, with a given resolution. This part of the code triggers bugs, for example the badly placed one way arrows. Gaël has tried to improve that part of the code, removing unneeded scaling. Difficult stuff that is not finished yet but Gaël is making good progress.

Thomas and myself have worked on the multi-page renderer. Following on previous Étienne work, we are trying to add the ability to split a big PDF map into several pages with a common index at the end. Yesterday, we spent the whole day trying to split correctly the original map into smaller pages that overlap with a small margin. Gaël found our bug late in the night and now the computation seems correct.

BTW, a big thank to leaflet: it helped us produce some quick debug code to show the computed bounding boxes.

Posted in hackfests, maposmatic, ocitysmap | Comments Off

MapOSMatic hackfest, start of day 5

Wednesday, March 28th, 2012

On Monday and Tuesday, we have started to work on more difficult bugs and elaborated features.

Thomas has tried to find why the one-way street arrows on the MapQuest style sheet are wrongly placed. In fact, he found a bug in Mapnik2. But there are still some strange things occurring with this style sheet. :-)

Sylvain and myself (David), we have worked on a new “Black & White” style sheet. This style sheet is a derived version of original OSM Mapnik style sheet that contains less colours, for example on roads or buildings. This style sheet should produce much more readable maps when printing on black and white printers.

Étienne, Thomas and Gaël have started to work on an old feature request: maps cut into several pages, with a common street index. This feature is complex and difficult to implement, thus there is not much to show right now. Stay tuned! ;-)

Posted in hackfests, maposmatic, ocitysmap | Comments Off

MapOSMatic hackfest, day 3

Monday, March 26th, 2012

We are now starting the third day of the hackfest and some progresses have been made.

First of all, we have fixed small bugs that where hiding in the corners:

  • The display on thumbnails was breaking the website with Django 1.3, this is fixed;
  • The port number to access the GIS (Geographical Information System) is now a configuration parameter;
  • We use right and left arrows, displayed in bigger size, to navigate in the Create map dialogue;
  • We display a In progress icon when somebody is typing characters to look for a city name, giving a better feedback to the user on what the website is doing;
  • The user can now navigate with the Nominatim results, with Next and Prev buttons;
  • We display a message that explains why no Nominatim result has been found and how to fix it;
  • Hide the right arrow when the paper size is loading in the Create map dialogue;
  • When the same amenity appears several times in the same grid coordinate (e.g. several building of the same town hall), it appears only one in the index.

We have also started to work on small feature requests. Until now, we have implemented:

  • The display, on both the website and the generated maps, of the most recent date at which the OSM data has been updated;
  • The display of villages and hamlets in the index, a useful feature in rural areas.

You can see the all those features on the current development web site (for France only).

We are also working on the set-up of a new world OSM database in a suitable format for this new version of MapOSMatic. That way, we hope to deploy this new MapOSMatic to the world before the end of this week.

Posted in database, hackfests, maposmatic, ocitysmap, website | Comments Off

MapOSMatic hackfest in Rennes, starting up

Saturday, March 24th, 2012

Just a quick note to let our readers know that a MapOSMatic hackfest is taking place this week, from March, 24th to April, 1st in Rennes, France. David Mentré, Gaël Utard, Frédéric Lehobey as well as myself, who have already contributed to MapOSMatic, will be joined by a few other developers for a full week of intense coding around MapOSMatic.

We will try to publish blog posts throughout the week to let our readers know about what we are doing and where MapOSMatic is going.

Posted in hackfests, maposmatic | Comments Off

MapOSMatic hackfest in San Francisco: what happened

Saturday, March 24th, 2012

DotCloud

On Saturday, 18th and Sunday 19th of February, a two days MapOSMatic hackfest took place in San Francisco, in the offices kindly provided by DotCloud. David Decotigny, Maxime Petazzoni, Jérôme Petazzoni and myself, Thomas Petazzoni, were the participants of this short session of hacking around MapOSMatic. The result of those two days of work is not yet visible on the production web site, but we hope to make it available soon.

Amongst the things that have been worked on:

  • David has worked on understanding a Mapnik rendering bug that happens with “halo” text when the Cairo surface is scaled and rendered to a PDF file. This bug is currently very annoying for MapOSMatic because all text rendered with a “halo” in the background looks really ugly on PDF maps. We have reported this problem in August 2010 on Mapnik bugtracker (see this ticket, which has then be converted to Github to this ticket). After two days diving into the Mapnik source code, David found out that disabling the glyph cache used by Mapnik made the problem go away, but this workaround would presumably significantly affect the rendering performance, so it really is a workaround and not a solution. We hope to get some feedback from the Mapnik community about this, but since last month when David worked on the issue, we haven’t so far received any feedback.
  • Jérôme has worked on the monitoring of our servers. He has set up a monitoring website at http://stats.maposmatic.org/r2d2/. Amongst classical metrics (CPU, disk, etc.), Jérôme has added one special metric: the lag that our OSM database has compared to the official OSM database. Many of our users have often been asking why they are not quickly seeing their OSM changes in MapOSMatic maps. The diagram is visible at http://stats.maposmatic.org/r2d2/browse/gcc10.fsffrance.org/replication/ and shows this replication lag in seconds. At the moment, we are about 1.2 million seconds behind the official OSM database, which means about 14 days of lag.
  • Maxime has worked on the infrastructure as well, migrating the development version of the MapOSMatic installation to Mapnik 2, installing the MapQuest stylesheet, updating the OSM stylesheet, starting a new GIS database for importing the OSM data under the new schema expected by the latest OSM stylesheets.
  • On my side, I worked on fixing various minor issues on the web frontend that are currently preventing us from putting in production the current development version.
MapOSMatic Hackfest in DotCloud offices, San Francisco

MapOSMatic Hackfest in DotCloud offices, San Francisco. From left to right: David Decotigny, Jérôme Petazzoni and Maxime Petazzoni. I am behind the camera :-)

All in all, in just two days, we made a bit of progress, but not a lot. None of us had touched the project for quite a while, so it took some time to get up to speed, and two days were not sufficient to bring major results. However, we knew that a second hackfest would happen in France a month after, so the work will definitely continue !

Posted in hackfests, maposmatic | Comments Off

PostGIS database is back online

Friday, January 20th, 2012

Our PostgreSQL/PostGIS database is finally back online and operational! We have rescheduled all the failed jobs since the beginning of the outage so renderings are generated for all of them.

Currently the database unfortunately has a 21 days replication lag with regard to current OpenStreetMap data. The replication process is back online as well but will take a while to catch up. We’ll keep you posted on the status of the replication.

Thank you for your patience and a big thank you again to the FSF France for giving us access to one of their server to host our database and for taking care of it when it goes down :-)

Posted in database, maposmatic | Comments Off

PostGIS database down

Monday, January 16th, 2012

The server hosting our PostGIS database, gracefully made available to us by the FSF France, is currently undergoing maintenance. This unfortunately means we can’t process any maps at the moment, but we’ve been assured that everything is put in motion to bring this server back online as soon as possible, hopefully in the next couple of days.

We’ll keep you updated on the status of the database server; in the meantime new renderings will be queued up and the failed renderings will be rescheduled to make sure all the requested maps are generated. It does unfortunately also means these renderings will represent outdated geographical data while the PostGIS database replication catches up.

Posted in database, maintenance, maposmatic | Comments Off

MapOSMatic database down

Tuesday, May 3rd, 2011

As you may have noticed, recent renderings have been failing on MapOSMatic. This is because our main PostGIS database is currently down. We are working on getting more information on the outage from the nice folks at FSF France who gracefully allow us to host our database on one of their servers.

All the jobs that failed because of the outage have been requeued, so they’ll be rendered as soon as the database is back up and we can restart the processing queue.

Hang in there!

edit: we’re back! The database server is back online and the rendering pipeline has been restarted and is now catching up on the queue. Thanks for your patience!

Posted in maintenance, maposmatic | Comments Off

« Older Entries