Background color of a button in CollectionView Data Template getting reset on scroll

I am setting the background color of a button on the Clicked event and it gets reset to its default color when the item is scrolled out of view.

Here is my CollectionView Data Template where the background color is initially set to Gray:

<DataTemplate x:Key="MyTemplate">
    <!-- Other controls above -->
    <Button Clicked="Handle_Clicked" 
              Text="Test" 
              BackgroundColor="Gray">
        </Button>
</DataTemplate>

Here is the event handle for that button:

void Handle_Clicked(object sender, System.EventArgs e)
{
    var button = (Button)sender;
    button.BackgroundColor = Color.Red;
}

This works and sets the background color to red. When this item is scrolled out of view and then returned to the background color gets reset to Gray.

I'm pretty sure it has to do with CollectionView virtualization - does anyone know how to maintain the background color of an item in a collection view while scrolling?