Is the study that used garbage studies worth studying?

(AP Photo/Desmond Boylan)

The results from a study were sent to me a number of months back and I’ve just had the chance to read them. In July, Firearm purchasing and firearm violence during the coronavirus pandemic in the United States: a cross-sectional study was released. Before I started to dig through the information, I had a certain lens that I intended on looking at this study through, however, that lens quickly got shattered. We’re going to cut right to some of the goodies at the end of the study first.

Despite concomitant increases in firearm purchasing and firearm violence nationally, the magnitude of the increase in purchasing at the state-level did not explain the magnitude of the increase in non-domestic firearm violence. There was an association between firearm purchasing and domestic firearm violence in April and May; these were the months when physical distancing was at its peak, indicating that risk for domestic firearm violence associated with excess firearm purchasing may have been exacerbated by increased time spent at home and in the context of pandemic-related stressors. However, the estimates for domestic firearm violence were sensitive to the inclusion of state-specific linear trends, suggesting that unmeasured, state-specific, time-varying confounders influenced our findings. Such confounders may include trends in drug and alcohol use or changes in access to domestic violence prevention and intervention services (e.g., access to courts or community-based organizations).

That part in there about domestic violence was left in, as to not accuse me of being inauthentic. The primary focus of the above cited is that the study found that even with the increase in firearm purchasers, it did not “explain the magnitude of the increase in non-domestic firearm violence.” 

Just so we’re on the same page, nowhere in the study is “domestic firearm violence” defined. We’ll all have to use our imaginations on what that means specifically.

The particulars surrounding that so-called domestic firearm violence really can’t be attributed to the uptick in firearm purchases, can it? Besides the “numbers”, there’s nothing to lean on. The study might as well have said that they can link the increase in so-called domestic firearm violence to weight gain. A whole lot of people gained weight in those early months of the pandemic, in fact some of us, myself included, have not given up on that trend.

What I’m pointing directly at though, is best summarized in the conclusion of the work:

Nationwide, firearm purchasing and firearm violence increased substantially during the first months of the coronavirus pandemic. At the state level, the magnitude of the increase in purchasing was not associated with the magnitude of the increase in firearm violence. Increases in purchasing may have contributed to additional firearm injuries from domestic violence in April and May. Results suggest much of the rise in firearm violence during our study period was attributable to other factors, indicating a need for additional research.

Additional research means more funding and grants, eh? See how that works?

What’s the problem? This study says that there really isn’t a correlation between the uptick in those buying guns and those using them violently. Correct. What’s problematic is both the sources the study used as well as the language they used. Right in the first sentence, I got derailed:

Firearm violence is among America’s leading causes of death and disability (Wintemute, 2015)[…]

All stop. Sorry, not sorry. How are we to take this study seriously with statements like that? That’s the leading causes of death and disability? Let’s take a look at what Wintemute had to say in their study since it was cited for us:

During the ten years from 2003 to 2012, the most recent year for which data are available, 313,045 persons died from firearm-related injuries in the United States (20). These deaths outnumber US combat fatalities in World War II; they outnumber the combined count of combat fatalities in all other wars in the nation’s history (41).

Look at the rabbit hole we’ve gone down. In the Schleimer-McCort-Shev et.al study, they started out talking about firearm violence. Wintemute’s introduction talks about firearm-related injuries. Those are two very different things.

Wintemute claims that 313,045 people died from firearm-related injuries between 2003-2012, a decade. That data is according to the CDC. I did fact-check that number, and it’s correct according to the CDC Wonder Search.

Of those numbers, 181,900 were suicides. 2,445 were “undetermined”. 3,627 were “Legal Intervention / Operations of War”. And 6,314 were deemed “unintentional”. Wintemute and Schleimer-et.al are talking about two different things. If the study we’re looking at wants to be completely transparent, we need to revisit Wintemut’s numbers neglecting the non-violent/accidental, self-inflicted, and or lawful use of firearms.

The number of firearm deaths in that decade should be 118,759, not 313,045. This of course is if the authors of Firearm purchasing and firearm violence during the coronavirus pandemic in the United States: a cross-sectional study wish to have their statements about violence supported by authentic data.

The statement on its face that firearm violence is the leading cause of death etc. is also fake and false. Look at the same date range and deaths due to heart disease. Narrowed down to just the results for Ischaemic heart diseases (clogged arteries), that came to 4,126,773 fatalities. I can see how the people looking at the numbers can get confused. Under half a million is quite close to over four million. So I get it.

The other issue with Wintemut’s numbers include the citations that the deaths outnumbered all US combat fatalities in all of the wars in our nation’s history. I’m not going to chase down their citation. We’ll get our own numbers. Looking at the American War and Military Operations Casualties: Lists and Statistics by the Congressional Research Service the numbers differ. 575,507‬ are the number of “battle fatalities” the United States of America suffered for all American wars from the Revolutionary War to the Gulf War. If we want to include any of the other “conflicts”, our numbers go up. War “events” from 1991 to present are not expressed in that 575,507 number.

Wintemute claimed that the 313,045 deaths are more than 575,507. I tried to study zero point energy once upon a time, given that, the fuzzy math being used here must go beyond that topic in complexity. Whatever cool calculating device they’re using over there I’d like to get my hands on. 575,507 does not include our conflicts post ‘91, the 2nd Gulf War, or Afghanistan (which went out with a bang). 

300 and 500 thousand, just like half a million and four million. Easy to confuse. Clearly, inauthentic numbers and statistics were appropriated and lent to this study. That’s just looking at one half of the first sentence of the study.

The next big roadblock I ran into is where they were getting their so-called “gun violence” data. The Gun Violence Archive (GVA). The GVA uses untrustworthy means of aggregating data. The “numbers’ they come up with are based off of news reports. My colleague Lee Williams did a few run-downs on the GVA and has proved the data to be unreliable, at best. One of Williams’s reports he noted interviewing the creator/curator of the site:

When I interviewed Bryant in July for our special report, I asked him if he actually believed that the average news consumer even considers domestic violence or gang warfare when they hear the term mass shooting. Bryant replied, “I don’t know. I know what we want to do is provide numbers and let the journalists, advocates and ‘congress critters’ look at the data, glean details and drill down on it.”

The topic of GVA’s statistics and whether or not they’re trustworthy did come up in the report. Directly from Firearm purchasing and firearm violence during the coronavirus pandemic in the United State:

Additionally, as GVA data are based on news reports and other public sources, records of firearm violence may reflect undercounts due to incomplete reporting to police and news coverage, perhaps particularly for nonfatal injuries and in recent months, as many local newspapers have lacked resources for reporting or shut down.

[…}

Despite these limitations, GVA is the most comprehensive real-time database of firearm violence, to our knowledge, and it has been used in prior research (Kim, 2019; Leibbrand et al., 2020). To bias our results, there would need to be similarly-timed differential changes across states in GVA or NICS reporting.

The time is now for a full-on media and academic alert. The authors of the study basically said that these other people used the GVA so it’s acceptable. It’s not acceptable. The only reliable numbers that can be used come from the FBI Uniform Crime Report. I understand the authors would not have had the FBI Uniform Crime Report for 2020 because at the time of the paper’s publication, July of 2021, the FBI report was not released yet. What the researchers should have done is just wait it out. They tap danced around this inconvenient truth by using the qualifying statement that the database is “real-time”, whereas the trustworthy data was not to be released for another two whole months!

This is garbage reporting and data collecting. This kind of thing needs to be called out every time it pops up.

It’s dangerous and frightening that scholars are leaning on the Gun Violence Archive to draw their data from! We need to demand that both our news media and academics reject the GVA and their unscientific means of collecting data. Follow the science? I think GVA does the opposite and leans on conjecture and hearsay. GVA is the data equivalent of a red flag report. Might be reliable, might not be, and either way, the damage is done!

These are just two instances of the report using untrustworthy and or outright wrong data. These statistics need to be gone over and really fact checked when citing the reports they’re in.

That statistic about wars, I’ve heard it before, but now we know. Go check out the report on casualties and do the math yourself as well. This way, the next time you hear a bunk number, you can cite the right one and point to the source of the real data. The last time I personally tried to do that math, the Congressional Report I cited was not in existence at that time.

Whether or not the authors of this study or the studies cited within purposefully were misleading is not known. What’s important though is setting the record straight, and I’ve done some of that here. What’s also very meaningful is that even though the data was wrong, misleading, erroneous, and or very biased, the authors of this study still were unable to link the rise in firearm purchases to the rise in violent crime during the pandemic. That’s something we could have told them. We know that gun ownership rose and crime decreased since the ‘90’s, except in the age of the pandemic. But, when do they really want to listen to “us”?

Author’s Note: Grateful appreciation for #mypeople #tealteamtwo for the story lead.

References:

Schleimer, J.P., McCort, C.D., Shev, A.B. et al. Firearm purchasing and firearm violence during the coronavirus pandemic in the United States: a cross-sectional study. Inj. Epidemiol. 8, 43 (2021). https://doi.org/10.1186/s40621-021-00339-5

Centers for Disease Control and Prevention, National Center for Health Statistics. Underlying Cause of Death 1999-2020 on CDC WONDER Online Database, released in 2021. Data are from the Multiple Cause of Death Files, 1999-2020, as compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program. Accessed at http://wonder.cdc.gov/ucd-icd10.html on Jan 28, 2022 3:20:08 AM

Wintemute, Garen J., The Epidemiology of Firearm Violence in the Twenty-First Century United Annual Review of Public Health (2015). https://www.annualreviews.org/doi/10.1146/annurev-publhealth-031914-122535#abstractSection 

Blum, David A., CRS Report RL32492, American War and Military Operations Casualties: Lists and Statistics (2020). https://sgp.fas.org/crs/natsec/RL32492.pdf