Billionaire investor Warren Buffett has set his sights on the highly fragmented U.S. auto sales business.

His Berkshire Hathaway Inc. holding company on Thursday agreed to buy Van Tuyl Group, the fifth-largest auto dealership firm in the U.S. with some 78 dealerships from Florida to California.

Mr. Buffett declined to disclose the deal’s…