This phrase is used to mean that one has had a very tiring or mentally-draining day. As for the “long” part, it usually means that that the day seemed to drag on forever. However, you can say “it’s been a long day” even when it is in the morning or afternoon.
For example, let’s say that you went to your job at 9:00 AM, and by 11:00 AM everything that could go wrong did go wrong at your work. If someone asked you how your day had been going, you could say “it’s been a long day” to suggest that it’s been a very tiring day, even though your day isn’t half way over yet.