Solved Error while removing from a list

Hi I created a HashSet (previously tried List) but both got me this error. I first thought it was because it had duplicated strings in the list.

Here is the error: (InvalidOperationException: HashSet have been modified while it was iterated over)

Code:

public HashSet<string> tempBlocks = new HashSet<string>();

Code:

foreach (var block in tempBlocks)
            {
                if (data.Blocks.ContainsKey(block.ToString()))
                {
                    data.Blocks.Remove(block.ToString());   //works
                  //tempBlocks.Add(block.ToString()); //this line works here and when I use it somewhere else
                    tempBlocks.Remove(block.ToString()); //brings error
                    //tempBlocks.Clear(); //brings error
                }
            }

I can Add, but not Clear or Remove :/ I suppose it's because im reading the block and trying to remove it at the same time? how can I remove it after using it for validate without having to do multiple steps?