Recently I was on a private mailing list thread where there was debate about whether or not the project should take on steps to improve diversity. One of the mailing list participants was very upset about this idea, and said that they didn't like when people accused them of the "original sin" of having white male privilege.
I suspect this is at the root of a lot of misunderstanding and frustration around the term "privilege". Privilege is not a sin... you are not a sinner for having privilege. However it is a responsibility and a debt to be repaid and corrected for, stemming from injustices in society.
A popular social narrative is that everyone has an equal place at the starting line, so the winners and losers of a race are equally based on their merit. Unfortunately this isn't true. Privilege is being able to show up at the starting line having had sleep and a good meal and the resources to train... you still worked to be able to get to the finish line and having privilege does not take that away. But if we look at the other people on the track we could see that they not only maybe didn't get enough sleep or were not able to allocate time to train (maybe they had to work multiple jobs on the side) or couldn't afford to eat as healthily. Some of them actually may even have to start back farther from the starting line, there are rocks and weeds and potholes in their paths. If we really want to treat everyone based on merit, we'd have to give everyone an equal place at the starting line, an equal track, etc. Unfortunately, due to the way the race is set up, that does mean needing to correct for some things, and it requires actual effort to repair the track.
My spouse Morgan Lemmer-Webber is an art historian and recently got into free software development (and software development in general). She has faced sexism, as all women do, her entire life, but it was immediately more visible and severe once she entered the technology space. For example, she wrote a web application for her work at the university. I helped train her, but I refused to write any code because I wanted her to learn, and she did. Eventually the project got larger and she became a manager and hired someone whom she was to train to take over development. He took a look at the code, emailed her and said "Wow, this file looks really good, I assume your husband must have written this code?"
What a thing to say! Can you imagine how that must have felt? If I heard something like that said to me I'd want to curl up in a ball and die. And she had more experiences like this too, experiences she never had until she entered the technology space. And if you talk to women in this field, you'll hear these stories are common, and more: dismissal, harassment, rape threats if you become too visible or dare to speak out... not to mention there's the issue that most of the people in tech don't look like you, so you wonder if you really actually belonged, and you wonder if everyone else believes that too. Likewise with people of color, likewise with people in the LGBTQ space... stones and disrepair on the path, and sometimes you have to start a bit farther back.
To succeed at the race with privilege, of course you have to work hard. You have to train, you have to dedicate yourself, you have to run hard. This isn't meant to take away your accomplishments, or to say you didn't work hard. You did! It's to say that others have all these obstacles that we need to help clear from their path. No wonder so many give up the race. And there comes the responsibility and debt to be repaid: if you have privilege, put it to good work: pitch in. If you want that dream of everyone to have an equal place at the starting line to be true, help fix the track. But there's a lot of damage there... it's going to take a long time.