This is a good one.
If the weapon is a firearm, I'd try and back out saying, "Oh crap! I just remembered I left my wallet in my car, then proceed to gather my stuff and leave.
If my shoes and other items were already through the xray machine, then I guess honesty is the best policy.
I'd ask to speak with a supervisor to the side BEFORE I went through the detector, hopefully in private, and explain myself and my mistake.
Depending of the state and laws, I suppose at the very least I face arrest for concealed weapons charges, and you never know, I might (doubtful) be able to get out of there sans handcuffs. If it's a knife, depending of the value, I'd either toss it, or leave the airport if I did not want to part with it.
If it's a BIG knife, or illegal knife or weapon, and I had no other option to not draw attention to myself as a suspicious person, I would again, come clean.
Not sure if any of these are the "best" courses of action, but one thing I do know, is that I would never be caught in that situation to begin with.
There's a bit of Metablade in all of us.