If I may borrow from a ton of vids I've seen about it...
Hard or very hard levels are levels where you have trouble getting through, and it's your fault for timing your jumps incorrectly or failing to notice something, I dunno.
Frustrating levels are levels where you have trouble getting through, and IT'S THE DESIGNER'S FAULT FOR MAKING THIS PIECE OF CRAP I MEAN HOW AM I EXPECTED TO MAKE THIS JUMP EVEN WHERE DID THAT ENEMY EVEN COME FROM WHYYYYYYYYYYYYYYYYYYY
So yeah. For example in Megaman you often find a new enemy, but you are not in its line of fire when it first attacks, so you can see how it attacks. Therefore, if you keep going and you get hit, it's your fault.
But in Megaman X6 to get through Gate's Fortress you'll need to acquire the Jumper part if you want to make it through, which allows you to jump a bit higher. This part is held by one of the reploids you save through the stages. You aren't told he has it, nor are told about the importance of getting some parts. And there's enemies called Nightmares which can kill these reploids permanently. As in, you can return to the stage and the reploids won't be there with the part anymore. So if you don't know about this, and you either skip past the reploid, or the reploid dies... then you can't get the Jumper part. Good luck getting through Megaman X6, because the game is unfair as hell.
I still like it thoughSo basically, it's about being fair to the player. Put as many traps as you want, as long as it's "fair". It's very easy to walk the line between hard and frustrating. You'll need experience to figure this out.
Bonus Q1: The way I see it... Kaizo levels aren't meant to be fun.

They are challenging. They are meant to put the players's skills to the limit and then some. Therefore, they will mostly only appeal to the challenge type of gamer.
Bonus Q2: When I am presented with clues as to how to continue, then it's fair. As in, I can see the platforms I need to go through (or notice them somehow if they're invisible or whatever), or I can figure out how to kill the enemies, and whatnot... I'm not sure of what to say of this.
For the record some people actually enjoy frustrating games. Meh.