Paul Gustard refused to throw in the European towel despite Tevita Cavubati's moment of madness that cost the London side victory in the Champions Cup.
Cavubati's rash off-the-ball tackle only a minute from time when he rushed out of defence and was penalised by referee Mike Adamson gave John Cooney the chance to win the game for Ulster.
It kept the Irish outfit top of Pool 3 with three successive wins but leaves Quins with just five points and needing to win all their remaining games to stand any chance of progressing to the next stage.
Craig Gilroy (left) and Ulster encountered Michele Campagnaro and Harlequins on Saturday
Stuart McCloskey was among the try scorers for Ulster in the Champions Cup Round 3 match
Ulster secured a narrow late victory, protecting their unbeaten home run in Europe since 2017
Quins head of rugby Gustard refused to point the finger of blame at Cavubati and instead insisted: 'It's so frustrating not to win, especially when it comes down to something like that at the end.
'But to come here and give Ulster the fright of their lives showed real courage and effort. They have not lost a European game at home since 2017 and we could easily have beaten them.
'It's not dead yet — we have Ulster at home next and we will be doing our level best to get the victory.'
Ulster were inspired by centre Stuart McCloskey who was a real force of nature, grabbing a try himself and being involved in most of their attacks.
McCloskey put Ulster ahead before two converted Elia Elia tries gave Quins a nine-point lead
Adam McBurney scored a try for Ulster with 13 minutes remaining as Quins let their lead slip
Luke Marshall (left) and Travis Ismaiel tussled before Cooney's decisive 79th-minute penalty
Cooney and Marcus Smith exchanged early kicking blows before England prospect Alex Dombrandt rounded off a move for the opening try for the visitors. The No8 took Kyle Sinckler's short pass and forced his way through two tacklers and across the line for a try converted by Smith.
Ulster drew level by the break with McCloskey catching and then off-loading Billy Burns' cross-field kick to flanker Sean Reidy who dived over for a fine try.
McCloskey put his side head shortly into the second half with a strong finish in the left-hand corner before the game appeared to swing in Quins favour with two quick tries from hooker Elia Elia.
Ulster were whipped into a frenzy by their passionate fans as first they mauled Quins back over their line with hooker Adam McBurney claiming the try. And then Cavubati lost his head as Quins tried to defend their lead and Cooney stepped up to seal the win and the points for Ulster.