Heat olive oil in a skillet over medium heat; cook chicken until slightly browned but still pink inside, about 5 minutes per side. Spread mustard-marmalade mixture over chicken; add blueberries to the skillet. Cook, stirring often, until chicken is no longer pink in the center, about 10 more minutes. An instant-read thermometer inserted into the center of a breast should read at least 165 degrees F (74 degrees C). Transfer chicken to a serving plate.
Pour vinegar into blueberry mixture in skillet; season with salt and pepper. Cook and stir blueberry sauce over high heat until 1/3 reduced, 5 to 10 minutes. Pour sauce over chicken to serve.