Firstly, you can never be too careful. Any remote hacking threat should be taken very seriously.
But even so, in the modern world of laze and lies, it's often the case that top-level access is breached by a hacker who isn't personally proficient enough to capitalise on the resources to which he's managed to acquire access.
It may be a trojan sniffer, who scanned for the signals of long-forgotten trojans on various corporate machines including your old one; or it may be the guy you sat next to in your local library when logging into your web host control panel.
These hackers are called chancers. They've been set up with privileged access, which a very good hacker could use to create backdoors to draw in bigger tools to maximise & extend access to useful, privileged data in a minimally conspicuous manner, which could result in the quiet theft of huge money; but this hacker does not know how to do anything more than install a bit of cheeky JavaScript into your main website's homepage, or to watch your keyboard impressions while you log into your email account and then to read and tamper with a few emails before you change your main passwords to deny any access that the hacker has obtained.
How do you explain to you non-technically-minded boss, that the threat imposed by the hacker is not something to lose sleep over?
Use the metaphor of an old-fashioned brainy bank robber. Say the guy who hacked into our system is like a guy who finds a set of keys for his local bank, and then uses them to rob the till for his christmas bonus, because he doesn't know how to use the keys to tackle the more complicated safe for enough money to comfortably retire on. He's a small-time chancer, who was lucky to received privileged access, which he has no clue how to thoroughly exploit.
