By Katy Burne 

The junk-bond boom has emerged as the biggest U.S. casualty of the oil bust, as a selloff in the debt of low-rated companies has punished investors whose thirst for higher-yielding debt fueled a wave of issuance.

Total returns on a popular high-yield bond index tracked by Barclays PLC fell into the red Tuesday for the first time since 2011. Junk bonds rallied Wednesday, likely taking the index back above water.

The shifts underscore the depth of a slump that began at midyear when oil prices were still above $100 a barrel. The average price on junk bonds is 96.4 cents on the dollar, down from full face value as recently as Dec. 4 and from 103 cents in September.

The declines have reversed a long period of junk-bond appreciation and threaten to crimp some borrowers' access to cash for expansion. The pullback is being widely scrutinized by analysts and portfolio managers who are on the lookout for any cracks in a five-year-long bull market in U.S. stocks.

So far, the signs are few and far between: The Dow Jones Industrial Average rose 288 points Wednesday to 17357 after the Federal Reserve reiterated its intent to be patient in raising short-term interest rates. Indeed, some investors are now scouring the rubble of the junk market for bargains, reasoning that much of the selling may be overdone and that the actual damage from the oil-price plunge is likely concentrated in energy companies' securities, which already have been hammered.

"I'm not worried about my Checkers [Drive-In Restaurants Inc.] bonds" just because oil is down, said David Sherman, head of high-yield mutual funds at manager Cohanzick Management LLC. "I think high yield is reasonable value."

Mr. Sherman said he has been reducing his fund's exposure to the energy sector all year but that the prices in the market are "starting to get interesting" enough to tempt him to buy again.

Still, the junk-bond swings highlight the risks buyers have embraced this year in a bid to generate scarce investment income. The risk hasn't paid off, as Barclay's U.S. Corporate High Yield index lost 0.3% this year through Tuesday's close.

That lags behind a 10.4% gain in the S&P 500, a 7.9% rise in investment-grade corporate debt, a 6% increase in the Barclays Aggregate bond index and a 5.5% rally in safe U.S. Treasurys.

To be sure, junk hasn't been the worst investment. The Russian ruble has lost nearly half its value against the dollar this year, and crude futures have dropped more than 40%. Emerging market stocks as measured by the MSCI Emerging Markets Index are down 8%, and U.S. small-cap stocks as measured by the Russell 2000 are up just 1% this year.

Still, this week marks the time the market has turned negative since 2011, a year in which junk sold off during the eurozone crisis before rallying to end the year up 4.98%. The last time junk bonds finished the year in negative territory was in 2008, when they lost 26% amid a broad flight from risk in the financial crisis.

Strong junk-market gains in past years have enabled low-rated issuers to raise more than a trillion dollars in the U.S. since the financial crisis, including a record $361 billion of issuance in 2013.

Some investors have been scouting for select high-yield bonds that others have jettisoned with the oil-price slump. Energy borrowings constitute 14% of the junk-bond market, according to Barclays' data, up from 5% in 2007.

"What surprised me in the last few weeks is many large asset managers were underweight high yield energy and so they are in a better position to step in here once thing start to stabilize," said Mark Pibl, head of high yield strategy at Canaccord Genuity Inc., a broker dealer.

J.P. Morgan Chase & Co. said that the recent junk price declines are the most severe since the Russian default crisis in August 1998, excluding brief periods around the 2001 terror attacks and the 2008 crisis. Over $200 billion of high yield bonds are now trading below 90 cents on the dollar, versus $115 billion at the end of last month and $57 billion as of January.

The premium investors can earn for holding junk bonds has risen to 5.52 percentage points above supersafe Treasurys, up from their 3.23 points in June and the largest so-called "spread" over Treasurys since late 2012.

Bonds from Chesapeake Energy Corp. due in 2022 traded at 94 cents midmorning Wednesday, up from 92.75 cents at the open of the session, and from 90.5 cents Tuesday, according to MarketAxess Holdings Inc.

Even so, investors broadly predict only a modest rise in defaults across the high-yield sector in the coming year and are sanguine about the health of corporate balance sheets.

"The down leg that we have taken has been really brisk," said Eric Gross, credit strategist at Barclays in New York, "but all of that feels very contingent on the price of oil."

Write to Katy Burne at katy.burne@wsj.com

Access Investor Kit for Barclays Plc

Visit http://www.companyspotlight.com/partner?cp_code=P479&isin=GB0031348658

Access Investor Kit for Barclays Plc

Visit http://www.companyspotlight.com/partner?cp_code=P479&isin=US06738E2046

Subscribe to WSJ: http://online.wsj.com?mod=djnwires