Instead, IRS employees were encouraged to process as many ID applications as possible, George said in his report. In addition, investigators learned that the IRS had quit using successful fraud-detection measures in processing the applications.

"There is a potential that erroneous tax refunds are going to non-qualifying individuals, allowing them to defraud the federal government of billions of dollars," investigators wrote in the report.

Once ID numbers are obtained fraudulently, they can be used for other deceptions. When first introduced in 1996, the ID numbers were supposed to be used only for taxes. But investigators found they're now being used in several states to get driver's licenses.

Almost 3 million tax returns seeking $6.8 billion were filed last year using IRS ID numbers rather than Social Security numbers. Investigators said they couldn't put a number on how many of those returns may have been fraudulent.