Dutch coach Louis van Gaal stunned onlookers at Salvador's Fonte Nova Arena by bringing on Krul to replace Jasper Cillessen in the final seconds of extra-time after 120 minutes had finished deadlocked at 0-0.
But van Gaal's audacious move to introduce Krul paid dividends as the Newcastle keeper saved Costa Rican penalties from Bryan Ruiz and Michael Umana to send the Dutch through into a last four meeting with Argentina in Sao Paulo on Wednesday.
Holland had won only one penalty shoot-out in five previous attempts at major championships.
But penalty takers Robin van Persie, Arjen Robben, Wesley Sneijder and Dirk Kuyt all made no mistake from the spot to beat Costa Rica goalkeeper Keylor Navas.
The unforgettable shoot-out drama came after a gritty duel which saw the Netherlands dominate for long periods without being able to score.
Sneijder came closest, twice hitting the woodwork in normal and extra-time.
The win means the Netherlands are set for another chapter of their World Cup rivalry with Argentina, which includes the 1978 final won by the South Americans.
Argentina advanced to the semi-final for the first time in 24 years on Saturday after a Gonzalo Higuain strike secured a 1-0 win over Belgium.
Napoli striker Higuain lashed in an instinctive first time shot after eight minutes as Belgium's hopes of qualifying for the last four wilted in Brasilia.
Argentina's win snapped a run of two successive World Cup quarter-final exits following failures against Germany in 2006 and 2010.
It is the first time the South Americans have qualified for the last four since the 1990 finals in Italy.
"We produced a very complete match. We weren't able to create that many chances, but they didn't make that many clear chances either," Argentina captain Lionel Messi said.
Argentina have now won all five of their matches at the tournament, but they laboured in the group phase and were criticised for a lacklustre display in their 1-0 victory over Switzerland in the last 16.
Beaten coach Marc Wilmots dismissively branded Argentina "an ordinary team with one extraordinary player" after the game, but Messi appeared similarly unimpressed with Belgium.
"All they had were long-range shots," he said.
"They have a lot of tall players, but couldn't do anything with them. We didn't give them space. The most important thing here is that we won and we got through."
Messi said that he and his team-mates took great pride in matching the achievement of Diego Maradona's 1990 side by reaching the last four.
"This is amazing," he said. "We knew it was going to be difficult.
"Argentina have gone a long time without doing this, and it was us who crossed the frontier."
Wilmots meanwhile paid tribute to his young side -- and delivered a withering verdict on Argentina.
"We made some very small mistakes, but we learnt a lot tonight. We're very disappointed, everyone is very down, but I'm very proud of the boys," he said.
"But I'm not at all impressed by Argentina. They're a very ordinary team with one extraordinary player," he said, referring to Messi.
A hard-fought contest at the Mane Garrincha National Stadium was settled in the opening skirmishes.
A ball from Real Madrid's Angel di Maria deflected off Belgium defender Jan Vertonghen and fell invitingly into the path of Higuain, who rifled a shot past Thibaut Courtois.
The win keeps Argentina on course for a possible dream final against hosts Brazil at the Maracana Stadium in Rio de Janeiro on July 13.
Match statistics Shootout: Netherlands 4 (Robin Van Persie G, Arjen Robben G, Wesley Sneijder G, Dirk Kuyt G); Costa Rica 3 (Celso Borges G, Bryan Ruiz NG, Giancarlo Gonzalez G, Christian Bolanos G, Michael Umana NG). Shots: Netherlands 20, Costa Rica 6. Shots On Goal: Netherlands 15, Costa Rica 3. Yellow Cards: Netherlands, Bruno Martins Indi, 64th minute; Klaas-Jan Huntelaar, 111th. Costa Rica, Junior Diaz, 37th; Michael Umana, 52nd; Giancarlo Gonzalez, 81st; Johnny Acosta, 107th. Offsides: Netherlands 13, Costa Rica 2. Fouls Committed: Netherlands 15, Costa Rica 13. Fouls Against: Netherlands 11, Costa Rica 14. Corner Kicks: Netherlands 11, Costa Rica 1. Referee: Ravshan Irmatov, Uzbekistan. Linesmen: Abdukhamidullo Rasulov, Uzbekistan; Bahadyr Kochkarov, Kyrgyzstan. Attendance: 51,179.