Here is a solution with alphabetically sorted tags on a single page.
It uses Liquid only, which means that it works on GitHub Pages:
{% capture tags %}
{% for tag in site.tags %}
{{ tag[0] }}
{% endfor %}
{% endcapture %}
{% assign sortedtags = tags | split:' ' | sort %}
{% for tag in sortedtags %}
<h3 id="{{ tag }}">{{ tag }}</h3>
<ul>
{% for post in site.tags[tag] %}
<li><a href="{{ post.url }}">{{ post.title }}</a></li>
{% endfor %}
</ul>
{% endfor %}
You can see it in action here.
EDIT:
There's also a way to generate a separate page for each tag without plugins (which will work on GitHub Pages).
I have a more detailed explanation on my blog:
Separate pages per tag/category with Jekyll (without plugins)
First, you need a new layout file:
/_layouts/tagpage.html
:
---
layout: default
---
<h1>{{ page.tag }}</h1>
<ul>
{% for post in site.tags[page.tag] %}
<li>
{{ post.date | date: "%B %d, %Y" }}: <a href="{{ post.url }}">{{ post.title }}</a>
</li>
{% endfor %}
</ul>
With this layout file, you can add a new tag page by adding a new file with just two lines of YAML front-matter.
Here's an example for the jekyll
tag:
/tags/jekyll/index.html
:
---
layout: tagpage
tag: jekyll
---
The only disadvantage of this approach: each time you use a new tag for the first time, you have to remember to create a new two-line file for it.
To generate the root index file (i.e. the list of tags that links to /tags/jekyll/index.html
etc.), you can use a similar solution like the one on top of this answer where I generate a single page with alphebetically sorted tags:
{% capture tags %}
{% for tag in site.tags %}
{{ tag[0] }}
{% endfor %}
{% endcapture %}
{% assign sortedtags = tags | split:' ' | sort %}
{% for tag in sortedtags %}
<a href="/tags/{{ tag }}/">{{ tag }}</a><br>
{% endfor %}
This will generate a list of links like this:
<ul>
<li><a href="/tags/.net/">.net</a></li>
<li><a href="/tags/authentication/">authentication</a></li>
<li><a href="/tags/backup/">backup</a></li>
</ul>
Note that this solution uses a blank to split tags, so it doesn't work when your tags contain blanks and Yevgeniy Brikman's comment applies here as well.