More than half of those workers are supposed to be ineligible to do work for the IRS because they are not enrolled in installment plans to pay the taxes they owe.
Unlike other federal agencies, the IRS requires employees and those who work on agency contracts to comply with federal tax laws. That means they have to file returns on time and either pay all the taxes they owe or enroll in a payment plan.
"Because many contractor employees have access to sensitive IRS systems and facilities, the IRS should address tax noncompliance for these employees in a similar manner as it would for its own employees," said J. Russell George, Treasury inspector general for tax administration.
The IRS does a good job of checking compliance when contract workers first start their jobs, the report said. But the agency should do a better job of monitoring whether workers continue to follow tax laws afterward.
The report said the IRS vigorously checks tax compliance among the agency's 90,000 employees. Contract workers should be held to the same standard, the report said.
"The IRS takes tax compliance for taxpayers and those who work for the IRS very seriously," the IRS said in a statement. "For an IRS employee, failure to timely pay one's full federal tax liability is considered misconduct, which may result in discipline or removal."
"With regards to contractors, the IRS remains committed to working with these employees to help resolve their tax liabilities, and we remain committed to strengthening our policies to ensure that contractor employees are and remain tax compliant," the IRS said.
The inspector general's office reviewed tax records for nearly 13,600 employees of IRS contractors. Investigators found 691 who owed back taxes as of June 2012, the report said. Some 352 of the workers owed back taxes and were not enrolled in a payment plan, for a delinquency rate of 2.6 percent. Those workers owed a total of $2.7 million in back taxes, the report said.
By comparison, the delinquency rate for all federal workers and retirees was 3.2 percent in 2011, according to the IRS statistics. The Treasury Department, which includes the IRS, had the lowest delinquency rate, at 1.1 percent.
Among the general public, 8.2 percent of taxpayers owed delinquent taxes in 2011.
IRS "employees are held to a very strict standard, even in cases of personal hardship. If they fail to file on time or pay their tax debts, they face disciplinary action, including removal," said Colleen M. Kelley, president of the National Treasury Employees Union, which represents IRS employees. "The IRS has the same requirement for contractors, but they are monitored with much less frequency."
The IRS said it will review cases of delinquent contract workers and take "additional action as necessary."
Wednesday's report looked only at employees of IRS contractors. The inspector general's office is working on a separate report on back taxes owed by companies that have been awarded IRS contracts, which is expected in the next several weeks.
In 2010, the inspector general reviewed tax records for 135 IRS contractors with contracts of $250,000 or more. At the time, investigators found 20 contractors that owed a total of $5.2 million in delinquent taxes.
Take ABC13 with you!
Download our free apps for iPhone, iPad, and Android devices