Contour maps have essentially gone out of fashion, replaced by shaded relief. There are still plenty of uses for contours, but they just don’t look as cool as a well-drawn hillshade. A colored hillshade is even more exciting. Traditionally, the easiest way to do that in ArcGIS was to place a color stretched DEM over the hillshade, and then reduce the transparency of the DEM. There are two major problems with this solution. One problem is that the transparency washes out the colors in the DEM, making the image too pale and bland. (This can be alleviated by choosing extremely bright colors, or adjusting the number of standard deviations in the stretch, but never quite solved.)The other is that when a transparent DEM over a hillshade is exported as a PDF (so that it can be edited in Adobe Illustrator), the alpha values are recalculated, making the image significantly darker.
Now, through the use of some deep image processing techniques, it is possible to brighten up your DEM overlays. The trick is to use pansharpening. Pansharpening uses a high resolution greyscale image to increase the color resolution of a multi-band color image. Don’t worry if you don’t exactly understand what’s going to happen – the wonderful folks at the ArcGIS mapping blog have put together a great tutorial. Within ten minutes, you’ll be wowing yourself with unbelievable shaded relief – without the hassles of transparency.
http://blogs.esri.com/esri/arcgis/2012/03/08/an-alternative-to-overlaying-layer-tints-on-hillshades/
Unless you’re working on a hobby project, as a cartographer, you will eventually be expected to show your work. And sharing means making yourself vulnerable to criticism. That word itself, ‘criticism’, sometimes conjures images of personal attacks and low self esteem, but that’s not always the case. Often, a criticism can be extremely helpful, both to you and to your organization. There are many different types of criticism, but only three of them will likely apply to your cartographic design: negative, practical, and constructive.
Negative
Example: “The legend looks terrible.”
Negative criticisms are the ones we’re really afraid of. Although they’re usually not meant to be personal, they can easily feel that way. Usually, you will only hear truly negative criticisms from people who actively dislike you, and people who are socially blunt. Most people have enough tact to offer practical or constructive criticisms. But don’t think the opinions of the tactless should be brushed away lightly. Even spiteful criticism can be helpful, if you can isolate the criticism from the spite. Poor feedback is better than nothing at all, because you can’t learn anything from nothing. For example, if someone snubs your legend, it might be a good idea to polish the legend a little, just to be safe. There’s always room for improvement, and a negative criticism can tell you where to focus for the maximum return on effort.
Practical
Example: “The legend is hard to read.”
Practical criticisms aren’t criticisms of quality, but of practicality. In a map, practicality is a function of readability and usability, so most practical criticisms will be very useful. As a cartographer, the map and the data seem quite clear to you. But an outsider, or someone seeing the map for the first time, won’t be as familiar with the material, and will be excellently positioned to see issues you hadn’t considered. Practical criticisms, even when they are blunt, are important to your work – which is why you should always find outsiders to critique your work.
Constructive
Example: “The legend looks a little crowded. Perhaps lowering the font size will help.”
A constructive criticism is a negative or practical criticism coupled with a suggestion for improvement. This format is how people are taught to discuss their work in school, because a learning environment is not the place to develop a thick skin – it’s a place to learn from mistakes. Therefore, criticism is cooled with helpful advice. The key for the professional cartographer is to separate the advice out, and focus on the criticism. In the example above, it’s possible that lowering the font size of the legend will help. But there may be other ways to uncrowd the legend, like streamlining the information presented, or repositioning text. Don’t think you are somehow bound to decrease the font size just because it was suggested constructively.
The opposite of criticism is praise. Obviously, you can never have too much of that, right? Actually, praise can be dangerous if it isn’t considered critically. If five people look at the map, and four people have a problem with the background color, are you going to trust the fifth just because her praise makes you feel better? Sometimes praise is just praise, but too much praise can mean you’re not asking the right people for feedback.
Ultimately, the goal is to make great maps, and even the best can get better. Any kind of feedback, whether, positive, negative, or neutral, can be useful to that end. So don’t be scared of criticism – seek it out! Find some friends or colleagues that you can trust to be honest, and ask for their help. And when other cartographers ask for your help, be honest but kind: give them the kind of feedback you’d like to receive.
Aileen Buckley, at the ESRI Mapping Center, has released a new tutorial on creating what she calls “sun glints”. These are the reflective glows you sometimes see on water bodies in aerial photographs. In essence, it’s a tutorial on creating gradient fills in ArcMap – a neat effect that cartographers don’t get to use very often.
The tutorial also includes instructions for randomizing the direction of the glints using Python. While her scientific explanation for this is sound, I’m not sure it’s really necessary. Personally, I wouldn’t use this effect to make a map more realistic, I’d use it to make a map look cool. Regardless, it’s a useful tutorial on a basic application of Python scripting.
This afternoon, I opened QGIS, and was greeted with a strange error message:
Couldn't load PyQGIS. Python support will be disabled. Traceback (most recent call last): File "", line 1, in RuntimeError: the PyQt4.QtCore module is version 1 but the qgis.core module requires version -1
QGIS will still open if Python support is disabled, but none of the geoprocessing tools will be available. The Vector menu won’t even show up! Clearly, this is bad news.
After a little research, I concluded that this version of QGIS (1.7.1 “Wroclaw”) was incompatible with the new PyQt4 library, which was passed to me as an update over the weekend. Eventually, a version of QGIS that is compiled against the new library will be available to install (and I hope that’s soon). But for now, the only solution for me is to downgrade my PyQt4 library from 4.8.6-1 to 4.8.3-2. This can be done from the command line:
sudo yum downgrade PyQt4
And QGIS is back to normal!
Note: this will only work in Fedora, RedHat, and other distros that use yum and rpm. For Ubuntu, you will have to use apt to perform the downgrade. See here for an example.
Slashgeo linked to an incredible web-based QGIS tutorial last week. I’ve been looking through it, and so far it is excellent. It’s not really designed for total newbies – there are other places to get your very first introduction to a desktop GIS. Instead, this tutorial seems to be aimed at those who are familiar with the concepts of GIS, but not yet familiar with QGIS. If that’s you, this tutorial will quickly get you up to speed. The combination of text instructions, screenshots (from a Mac!), and YouTube videos creates a fun and fast-paced learning environment which is ideal for people who want to get started quickly.
The only obvious drawback to QGIS presented here is the installation of the required software. Since QGIS was designed as a package for the Unix environment, it has a lot of dependencies on other projects, notably GDAL. And those packages have to be in place for the installation to proceed. Even the installation of Mac binaries requires a lot of hair-pulling. Windows standalone binaries are easy enough to install, but a static installation is not as extendable as a linkable application framework. So if you really want to capture the full power of QGIS, some assembly required.
My experience installing QGIS, in Fedora Linux 15 (and 12-14) has been much simpler. Fedora has a graphical package manager that lets me install QGIS just by checking a box (Figure 1). All the dependencies and prerequisites are taken care of by the package manager behind my back. Installing software doesn’t get much easier than that. Installing QGIS in Linux Mint, a spinoff of Ubuntu, was similar. Linux is still not the easiest OS to work with, but some distros have made a lot of progress on the package management front.
Figure 1: QGIS in the Fedora package manager
Link to the tutorial:
http://cga-5.hmdc.harvard.edu/qgis/
Chances are, you’re not familiar with OpenStreetMap. The market saturation of Google Maps and Bing Maps is such that they’re the tools we turn to, sometimes without even thinking. Both these services have an incredibly beatiful and usable design aesthetic, which makes them easy to read. There’s no real reason why you’d need a third web mapping service – unless you really wanted one. If you like having alternatives, especially alternatives that are open source, crowd-editable, and not controlled by nearly monopolistic companies, then OpenStreetMap may be the project for you.
The best thing about OpenStreetMap, besides the ability to add your own map features via a wiki-like interface, is the wide variety of viewing options. As you may know, web mapping services work by serving image tiles to your browser. The visual aspects of the map are dependent on which set (‘deck’) of tiles you are loading. Google Maps, for instance, has map tiles, satellite tiles, and topography tiles. OpenStreetMap currently has five different tile decks to choose from, using the handy blue layer switcher in the upper right corner of the map. Each deck has some overlap in functionality, and each has its benefits and drawbacks. To illustrate the differences, I’ve included links to the main campus of the University of Colorado at Boulder, my alma mater.
Mapnik
This default deck features the same basic light gray and muted yellow as a Google map, but with a wider pastel pallete to color-code buildings and street types. The inclusion of sidewalks is a nice touch, but the map feels like maybe it needs a key. One major drawback of this deck is that the campus boundary isn’t as clear as in other decks.
http://www.openstreetmap.org/?lat=40.0057697296143&lon=-105.267241001129&zoom=16
Osmarender
The streets and street names in this deck pop a little bit more, but the strong purple background on the campus makes it feel too busy. In some cases, the placement of the street names makes the map harder to read, especially where a trail runs close to the street, or a highway has a local name. But in less dense areas, these tiles are very easy to read, and would look great on a small screen.
http://www.openstreetmap.org/?lat=40.0057804584503&layers=O&lon=-105.267241001129&zoom=16
Cycle Map
Although the functionality of this deck caters to the needs of cyclists, it also looks great as a car or pedestrian map. The bluish gray looks excellent as a base color, which makes the bike lanes and paths easy to find. This map still feels like it needs a key, though: it’s not easy to tell that the yellow areas represent auto parking lots (which are bicycle hazards).
http://www.openstreetmap.org/?lat=40.0057697296143&layers=C&lon=-105.267241001129&zoom=16
Transport Map
Everything fades into the background except for the bus routes. It’s not easy to tell which buses follow which routes, but there is no question which streets have bus traffic. Some of the major intersections are labeled. which can be convenient for visiting riders. Personally, I would like to see the actual stop locations. Assuming that the tiles are constructed using open transit data (some of which uses the Google transit specification), that should be easy enough to add.
http://www.openstreetmap.org/?lat=40.0057911872864&layers=T&lon=-105.267241001129&zoom=16
MapQuest Open
Recently, MapQuest opened up their tiles to OpenStreetMap, and the result looks fabulous. This deck may not be as informative as other decks, but it is simple enough that it doesn’t really need a key. I’ve been having trouble with the tiles loading slowly, but that’s my only complaint. A very usable map.
http://www.openstreetmap.org/?lat=40.0058019161224&layers=Q&lon=-105.267241001129&zoom=16
Google Map for comparison:
http://maps.google.com/maps?hl=en&ll=40.005544,-105.267498&spn=0.0119,0.021865&vpsrc=6&t=m&z=16
One of the keys to good database planning is correctly anticipating what type of data each field will hold. If a field is expected to only hold integers, it is best to make it an integer field instead of a float or double field. Doing so will save space and time, and the database structure will be more intuitive. Likewise, it is a good idea to fix the number of characters allowed in a string field ahead of time.
Sometimes, though, you may run across data that is incorrectly typed. The most common error is a value that looks like a number but is actually a string of numeric characters – they may look the same, but they certainly don’t act the same. Models, scripts, and geoprocessing tools will go nuts when they try to do math on mistyped fields, and the vague errors thrown may take hours to decode. (I suspect these errors are caused by importing incorrectly typed MS Excel spreadsheets into a database, but don’t quote me on that.)
Fortunately, there’s an easy way to dynamically change the field type in ArcGIS. This is called typecasting, and it is a common feature in untyped programming languages like Python and JavaScript. To typecast a numeric string as an integer, create a new integer field. Then use the field calculator to multiply the numeric string by 1. The field calculator is “smart” enough to know that it can’t multiply a string, so it multiplies the number value of the string instead. The new field will then be populated with integer values that you can manipulate to your heart’s content. The same procedure can also be used to typecast numeric strings (or integers) as floats or doubles. Just multiply the values by 1.00 instead of 1.
Part of the GIS Day 2011 festivities at Texas A&M University included a student poster display in Evans Libary on the main campus. There were 29 posters covering a wide range of GIS topics from various classes and departments. All 29 posters contained a scalebar, a legend, and a North arrow. But only three contained any sort of projection information.
GIS analysis, by definition, requires 2 or more datasets – one dataset is just a map, right? For a spatial analysis to make any kind of sense, there has to be some level of confidence that the datasets are correctly aligned with one another. That confidence is easiest to achieve by providing projection information. Claiming that a map uses Wisconsin Central State Plane doesn’t prove that the data is properly aligned. But it does imply that the cartographer has taken every possible step to insure the correctness of the map. Knowing the projection is the first step toward correctly managing projections.
I wonder if providing projection information would be more common if desktop GIS layout programs used a simple function to insert it into the map? ArcGIS has made adding a North arrow to a project so easy that most users don’t bother to stop and consider whether it is functionally necessary. I’d like to see similar functionality with regard to projections.
Happy GIS Day everyone! GIS Day (and Geography awareness week) has certainly gotten more popular over the past year. The first time I ever heard of it was last year, when a former professor mentioned it on Facebook. This year, it seems like everyone is celebrating. Universities are hosting student poster displays, and local governments are hosting open houses and demonstrations. The economy has not been kind to the industry, so it’s nice to see that everyone has been able to keep their spirits up.
GIS Day is a special day at Northwest Spatial Solutions – it’s a birthday. One year ago today, I registered this domain and made my first post. At that time, I had been living in Hawaii for five months, and had just finished writing my first web-based geocoding application. I wanted to share it with someone, and I decided the internet would make a fantastic audience. (That geocoding application has also gone through some changes, and the newest version will be rolled out sometime next week.) And now here we are, 30 posts and 365 days later.
I guess this is as good a time as any to let everyone know that Northwest Spatial Solutions will be heavily revitalized over the next few weeks. You may have already noticed the new, cleaner look to the site – that’s just the beginning. Starting next week, I will be submitting two posts per week, most likely on Mondays and Wednesdays. I will also be promoting new posts via Twitter, Facebook, Google+, and LinkedIn. Over the next few months, the Portfolio section of this site will be filled in, allowing anyone to keep an eye on what I’ve been up to.
Today I’ll be attending a few conferences on the Texas A&M University campus. I hope you’ve got some celebrations planned as well.
When loading data into a PostGIS database using shp2pgsql, have you ever gotten this cryptic error?
Unable to convert data value to UTF-8 (iconv reports "Invalid or incomplete multibyte or wide character"). Current encoding is "UTF-8". Try "LATIN1" (Western European), or one of the values described at http://www.postgresql.org/docs/current/static/multibyte.html.
This means that there is an invalid or incomplete multibyte character in your data somewhere. Unfortunately, the error message figures the only possible reason for this is that you have got the character set of your database wrong. There is another possibility, though – that you got the character set right, and your shapefile contains an invalid multibyte character. Which is unfortunate, but easy enough to fix.
Open QGIS, and add your shapefile, making sure to set the file’s encoding to UTF-8. Now open the attribute table and start scrolling through the records. If it was displayed in octal or hex form, the invalid character would be impossible to find simply by scanning. Fortunately, QGIS displays invalid characters using an easily recognizable dingbat:
Once you’ve found the offending character(s), simply delete it, and replace if necessary. After closing QGIS, the shp2pgsql operation should complete without errors.