One often overlooked but highly important aspect to consider when building a new website is your link structure. Changing the link structure on an established site can result in broken links, so try to pick something you'll be happy with for a long time and only make changes when you really have to. So what's a permalink? Permalinks are links that will typically always point to a specific piece of content, unlike a category or tag link where the article might get pushed off the first page.
Rather than having a link like http://refactored.co/?p=80 which is enough for a server to know what to display (and handy if we need a short URL), we want our links to look something like http://refactored.co/blog/understanding-wordpress-permalinks so that real people can know the contents of a link before clicking it. WordPress uses rewrite rules to translate these human-readable links into which page needs to be displayed.
Setting up your link structure is as easy as going to Settings → Permalinks and choosing from some of the common settings or entering your own. If you choose to create your own, there are several tags you can use in the structure like %postname%, %year%, %category%, and %author%. Once you have your structure set up, save the changes. Usually this is all you'll need to do, but WordPress may give you further instructions about updating your .htaccess file, so I recommend checking out this guide if you're having any issues.
You may have read warnings in other tutorials to always begin your permalinks with a number (like the year or post ID) to avoid performance issues, but as of WordPress 3.3 this issue has been fixed.
You should now see that the links on your site are using the format you've chosen, but you may want to make some additional changes to the slugs for certain post names, categories, or tags. A "slug" is the link-friendly version of a title, which means that it's all lowercase, doesn't have special characters, and uses dashes instead of spaces. These get generated pretty well automatically, but occasionally you'll find a reason to tweak them. Sometimes I'll use an ampersand (&) in the title of a post, but because it's a special character it gets removed from the slug. In these cases I'll manually edit the post's permalink to include the word "and." For example, a post titled "Cats & Dogs" would have the permalink's slug automatically set to cats-dogs, so I'd change it to cats-and-dogs. If you want to change the slug for a category or tag just visit the respective section in the WordPress dashboard. There you'll be able to edit an item and customize the slug.
The last thing to know is that simply visiting the Settings → Permalinks page will flush the rewrite rules. This is something that you probably won't need to do often if ever, but if you're getting 404 errors after creating a new custom post type or installing a plugin that uses custom URLs you may need to do this. Typically plugins that need to do this will flush the rewrite rules when they're being activated or deactivated, but it's worth knowing this trick in case a developer forgot to automatically do so. I've had to waste time remembering this myself a few times after creating a custom post type.
I'll be writing an advanced tutorial soon for developers looking to create their own rewrite rules, but for know let me know if you have any other questions or tips about permalinks.
More information on using permalinks is available on the WordPress Codex.