Repeat Step 2 repeatedly to build a graph of activity surround | Computer hardware and networking

Repeat Step 2 repeatedly to build a graph of activity surrounding the compromised machine, a network of relationships related to the attack.
As you execute step 3, to keep the scope reasonable, you prune out machines that seem to have no sign of malicious activity. Depending on the network's capabilities you may also be able to identify other instances of the malware or other indicators that machines are compromised and start new searches from these new machines. You know you're missing something if the graph isn't fully connected.
At some point, you may find "patient(s) 0", the ingress point for the attacker, with some sign as to how they got in such as a phishing email or exploit. There may be more than one ingress point. On the other hand, sometimes reports of phishing can tip you off to the campaign in the first place and you'll know exactly how the attacker got in.
At some point, you will likely find that the malware is talking to a command-and-control server living outside your network. You will need to work with the owner of that server and possibly the legal authorities to find out what it's talking to. This is slow, and you may not be able to do it at all, either due to uncooperative owners or locales, Tor nodes, or because the attacker moves the server. This is usually where the trail goes cold. However, in the best case, you can access the logs of that machine forensically and figure out which machines it's talking to. Repeat that until you get blocked or find the actual attacker.
Separately there are ways to identify information about the attacker without actually tracing them. For example, if their active hours correspond to the working hours of a particular locale, the attacker may be working there (or may be trying to make you think they do). If they execute scripts, in what language are the scripts named, and if you can recover one, what language are the comments in? Are the malware and tactics similar to those used by a previously known attacker? They often share/steal from each other, but groups tend to have distinct MOs. For example, one group may strongly prefer to get Domain Admin rights and create lots of back doors to ensure persistence, while another group may compromise only the accounts they need for a particular goal and never write malware to disk, in order to minimize the chance of discovery. Also, their vulnerabilities tend to follow patterns - one group may like font exploits while another prefers Flash. Just like any other organization, individuals in attack organizations develop specialties and these specialties determine the organization's operational preferences.

Another angle: what were they trying to steal and who would want that? Often you can identify the group very precisely by the combination of tactics, malware reuse, and goals. An obvious case is Stuxnet: only a very limited number of groups have both the skill to create such a piece of malware and the desire to target Iranian nuclear facilities.

Keep in mind that even if a defender identifies the attacker by name, address and phone number, they may not get arrested if they live in a jurisdiction that is unfriendly to the victim's country or that has high corruption. And since many attackers are employed by their government, those guys aren't going to jail either, though I suppose they might get fired for having gotten caught :).

By telling everyone they hacked something
Using their real IP address (Which is very stupid and only script kiddies do that)
Leaving behind a trace
Not using strong enough anonymity software
If they know that they had used weak anonymity software than not wiping everything on their hard drive.
Selling the stolen goods with their real name
Not deleting the logs
Leaving behind a backdoor that could lead back to you