Are there any holistic remedies for getting rid of cold sores and keeping them at bay?

In fact reSource-therapy is absolutely right because 60 to 70 percent of immune system is located in the gastro-intestinal tract. A cold sore is a viral infection that your immune system normally keeps suppressed. When your immune system is compromised (such as when you are under extreme stress) these opportunistic organisms resurface. The best way to "get rid" of cold sores is to boost your immunity. You can do this from many different levels. The way you deal with stress is actually more important than the amount of stress in your life. Building stress coping skills can help prevent many diseases. Herbs such as Echinacea (which is immuno-modulating) and St. John' Wort (which is antiviral) can also help prevent and treat cold sores respectively.
Good nutrition is necessary to maintain good immune health. Also, cholustrum has been shown to boost immunity and prevent flus better that the flu vaccine itself. Quote below:
Clin Appl Thromb Hemost. 2007 Apr;13(2):130-6.
Prevention of influenza episodes with colostrum compared with vaccination in healthy and high-risk cardiovascular subjects: the epidemiologic study in San Valentino.
Cesarone MR, Belcaro G, Di Renzo A, Dugall M, Cacchio M, Ruffini I, Pellegrini L, Del Boccio G, Fano F, Ledda A, Bottari A, Ricci A, Stuard S, Vinciguerra G.
San Valentino-Spoltore Vascular Screening Project, Department of Biomedical Sciences, G D'annunzio University, Chieti, Pescara, Italy.
The efficacy of a 2-month treatment with oral colostrum in the prevention of flu episodes compared with antiinfluenza vaccination was evaluated. Groups included healthy subjects without prophylaxis and those receiving both vaccination and colostrum. After 3 months of follow-up, the number of days with flu was 3 times higher in the non-colostrum subjects. The colostrum group had 13 episodes versus 14 in the colostrum + vaccination group, 41 in the group without prophylaxis, and 57 in nontreated subjects. Part 2 of the study had a similar protocol with 65 very high-risk cardiovascular subjects, all of whom had prophylaxis. The incidence of complications and hospital admission was higher in the group that received only a vaccination compared with the colostrum groups. Colostrum, both in healthy subjects and high-risk cardiovascular patients, is at least 3 times more effective than vaccination to prevent flu and is very cost-effective.

During my teenage years and early twenties, I was plagued by coldsores - not helped by the fact I played in a brass band! My GP suggested I wear lipstick/salve on a daily basis to reduce minor trauma/cracking of the lip tissue where the herpes simplex virus lies dormant. Many moons later (too many!) I can verify that it does certainly reduce the number of coldsore infections.