Fish with Green Olives and Grapes

Name: Fish with Green Olives and Grapes
Ingredients: 2 pounds Yukon Gold potatoes; 1/3 cup heavy cream; 1/2 teaspoon granulated sugar; 1/2 tablespoon kosher salt, plus more for seasoning fish; 1 teaspoon black pepper, plus more for seasoning fish; 4 filets of white fish (I used walleye); 1 1/2 teaspoons dried tarragon; 1/2 cup flour; 6 tablespoons butter, divided; 1 shallot, minced; 3/4 cup red seedless grapes, halved; 1/2 cup Castelvetrano green olives, crushed with the side of a knife to pit and half; 1/2 cup dry white Vermouth (I use Cinzano); 2 tablespoons fresh tarragon leaves, chopped