President Joe Biden is issuing an executive order pardoning all Americans who have been federally convicted of possessing small amounts of marijuana.
The move will only affect a relatively small number of cannabis convictions since most occur at the state level.
Mr Biden's announcement stops short of a full decriminalisation of pot, which he promised as a presidential candidate.
Recent polls show that a majority of Americans believe weed should be legal.
"Sending people to prison for possessing marijuana has upended too many lives and incarcerated people for conduct that many states no longer prohibit," Mr Biden said in a statement on Thursday.
He added that racial minorities were statistically far more likely to be jailed for cannabis.
As a White House candidate, Mr Biden was criticised for writing a 1994 crime bill that stiffened penalties for drug crimes and led to more incarceration of racial minorities.
In a briefing on Thursday, a White House official told reporters there are "no individuals currently in federal prison solely for simple possession of marijuana".
However, the pardon will remove "needless barriers to employment, housing, and educational opportunities" after convicts' records are wiped clear, Mr Biden said.
US officials estimate about 6,500 people with federal convictions for simple possession of marijuana will benefit from the Biden pardons.
The Democratic president said he would call upon all state governors to issue their own marijuana pardons.
He is also directing the Department of Justice and the Department of Health to review how cannabis is classified under federal law.
"We classify marijuana at the same level as heroin - and more serious than fentanyl," said Mr Biden. "It makes no sense."
Recreational marijuana is already legal in 19 states and Washington DC. Medical use is legal in 37 states and three US territories.
However, the drug remains illegal at the federal level, even in states where it can be legally bought and used by millions of adults.
The pardons come a month before November's congressional mid-term elections, which will determine the power balance in Washington for the last two years of Mr Biden's term.
Life for Pot, a website advocating for the release of non-violent marijuana offenders, noted that there are no known federal prisoners that will be affected by Mr Biden's measure, tweeting: "This is window dressing."
Cannabis company shares jumped on the stock market by around 20% with news of Mr Biden's pardons.
Mr Biden is not the first US president to pardon cannabis offenders.
On his final day in office, Donald Trump pardoned 12 marijuana offenders, including some who had been jailed for life under the three-strikes rule created by Mr Biden's 1994 crime bill.