Coronavirus

[cherdano]

The Pfizer trial is already beyond 164 cases. But 164 cases was deemed the appropriate number to prove effectiveness of a vaccine that works 60% of the time. For the observed efficacy, a much lower number of cases would have been enough.

 

Just yesterday, the WHO announced the end of an Ebola outbreak in Congo, thanks to a vaccination campaign. The vaccine - needs to be stored at -80C.

[thepossum]

The Pfizer trial is already beyond 164 cases. But 164 cases was deemed the appropriate number to prove effectiveness of a vaccine that works 60% of the time. For the observed efficacy, a much lower number of cases would have been enough.

 

Just yesterday, the WHO announced the end of an Ebola outbreak in Congo, thanks to a vaccination campaign. The vaccine - needs to be stored at -80C.

 

Thx. As I said, I just found the Nature article and was checking on the numbers related to above discussions

 

Relating to Ebola though, we are not talking about vaccinating a whole planet in quick time, so I imagine it has proved practical to have the mobile refrigerated units (or however it is done) for relatively small outbreaks of a very different virus type

[pilowsky]

My understand of the Moderna results is that they enrolled 30K people in a trial.

Some of the folks in the trial received a placebo, others got the actual vaccine.

 

Of the folks who were enrolled in the trial, 90 of them ended up getting Coronavirus.

So, 1:333 participants in the trial contracted COVID

 

When they went and looked at who precisely contracted COVID, they found that

 

85 of the people who contracted COVID received the Placebo

5 of the people who contracted COVID received the vaccine

 

85/90 = 94.4%

 

The key issue here is the relative small number of individuals who ended up contracting COVID regardless of whether or not the received the vaccine.

 

With only 1 every 333 participants contracting the disease, you need an enormous sample size to boost those numbers significantly.

 

Or, alternatively, you need to start vaccinating people and then deliberately exposing them to COVID.

(However, this is an incredibly dicey subject which raised all sorts of moral quandaries)

 

One good thing is that the first two vaccines were both using similar techniques (mRNA) and both are showing consistent results...

 

 

Well, there's the rub,

Since coronavirus is the agent and COVID is the disease, and the participants in the trial are a quite different population to the people that are out there in the 'real world' where Doctors treat patients, that's where the rubber hits the road.

 

Thank you all for clarifying the maths.

 

Following the (media) commentary, there is now a major structural problem getting the vaccine to people who need it for many reasons, and then getting them to take it.

Why is compliance a problem? Just to give an example, cigarette smoking causes 480,000 deaths every day in the USA. Alcohol causes 95,000 and so on. One of the main reasons that COVID-related deaths have hit the USA and the UK so hard is a failure to 'believe' in it.

Along with pre-existing morbidities many of which are self-inflicted (diet/cigarettes).

Confidence intervals can't overcome this.

 

It may seem that I am changing the topic, but it does as I say relate to the original highly motivated study participant population.

 

Speaking of which, does anyone know how many of the participants completed the study? 15000/15000 is an amazing result for a clinical trial. One normally loses at least one or two people to follow-up.

 

 

 

 

 

[barmar]

I'm not a statistics expert, but I think the real point is that developing a COVID-19 vaccine is extremely urgent. Normally it takes several years to develop and test vaccines, this allows you to wait for many results and have much more confidence in the results. But we need something ASAP, we can't wait for thousands of trial participants to die.

 

But there's also more to it. This is Bayesian. We have prior expectations based on the theory behind the vaccine design, and 90 vs 5 is significant enough to confirm the hypoethesis, even though they're both small compared to the group sizes of 15000. It's unlikely to be a statistical fluke since we have good reason to believe that the vaccine will be effective.

 

As an analogy, suppose you're trying to determine if a die is fair. Ideally you would count the number of all the different faces, and see if they're approximately equal. But if you're already pretty sure that it's a fair die, and you just want confirmation, you could get away with counting just one of the faces. If it's close to the expected fraction of rolls, you'd consider your hypothesis confirmed. Not as well as the first method, but good enough.

[pilowsky]

That point is completely true. It really does look like an effective vaccine in the group that it was tested on.

Let's hope that it also works when it gets distributed, that the supply chain works, that people that get very big doses of the virus are also protected and so on.

 

It is extremely promising. The data was analysed appropriately. It is a good start.

 

None of that means we stop asking questions, thinking or taking precautions.

 

Look how difficult it was to eradicate polio, TB is still a massive problem. To say nothing of malaria etc etc.

 

I get it, it's a Bridge Forum so there is a natural tendency to focus on probability and restricted choice (Bayes theorem) which is where my research career started.

I'm just saying that there is much more to it. Developing a vaccine is important, but it is not the most important.

 

Just compare the experiences of different countries around the world.

[pilowsky]

I used to live in Adelaide - South Australia. Near where Howard Florey was born. They are in another lockdown because of a 17 case cluster.

They are experiencing major problems. The local music scene is in trouble, and there's this.

 

http://i.imgur.com/4YUUKmf.png

[barmar]

I think it's also instructive to compare modern methods to the development of the Jenner smallpox vaccine. I just reviewed the latter's history at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1200696/

 

Basically, he'd heard that dairymaids who suffered from cowpox (a relatively mild disease) rarely contracted smallpox. He found a dairymaid with cowpox, took some of the material from one of her lesions and innoculated a young boy with it. He got sick for a short time. A month later he innoculated the boy again, and this time he didn't get sick. He then tried it with a handful more subjects, and concluded that it worked. He published his results and vaccination became popular among other doctors. After this had been going on for a while he surveyed people who had been given vaccinations, found that they were generally protected from smallpox, and considere this confirmation of the earlier results.

 

That's it. No double-blind trials, just a handful of tests and a bunch of anecdotal evidence. And not much in the way of safety checking, they were mostly interested in effectiveness.

 

Most modern drug and vaccine development is the polar opposite of this method. It's a slow process, but generally effective (but there have been some notable failures, like thalidomide and Dalcon Shield). For COVID-19, we're clearly aiming somewhere between these extremes because of the urgency.

[pilowsky]

Interesting that you mention thalidomide.

Thalidomide was one of the great success stories of the American food and drug administration FDA.

It was the rigour of the FDA in making sure that it was properly tested that prevented it from entering the US market, which is why the US has no thalidomide victims to speak of.

 

Imagine if the President at the time was touting it as a wonder drug?

 

Access to internet pharmacies and illegal drug streams that can bypass such controls are now a major problem.

They are even lauded by Hollywood movies.

 

My first research thesis was examined by the woman whose work was the scientific basis for understanding how thalidomide causes birth defects.

Her name is Dr Janet McCredie a Sydney radiologist. Not the discredited Dr McBride - but that's another story.

She was very kind, but warned me to watch out for 'eros in the poofs' - smerriman would have been proud.

[thepossum]

I'm not a statistics expert, but I think the real point is that developing a COVID-19 vaccine is extremely urgent. Normally it takes several years to develop and test vaccines, this allows you to wait for many results and have much more confidence in the results. But we need something ASAP, we can't wait for thousands of trial participants to die.

 

But there's also more to it. This is Bayesian. We have prior expectations based on the theory behind the vaccine design, and 90 vs 5 is significant enough to confirm the hypoethesis, even though they're both small compared to the group sizes of 15000. It's unlikely to be a statistical fluke since we have good reason to believe that the vaccine will be effective.

 

As an analogy, suppose you're trying to determine if a die is fair. Ideally you would count the number of all the different faces, and see if they're approximately equal. But if you're already pretty sure that it's a fair die, and you just want confirmation, you could get away with counting just one of the faces. If it's close to the expected fraction of rolls, you'd consider your hypothesis confirmed. Not as well as the first method, but good enough.

 

I wouldn't claim to be a statistics expert either and my peers have not recognised me at that level. However I thik due caution is still due irrespective of all the above. I also have serious concerns (without suggesting impropriety) about a certain wild west approach to biotech during 2020. I also find such small numbers worthy of due suspicion/challenge/question/diligence/whatever

 

And yeah things may be Bayesian but we started off knowing absolutely nothing only a few months ago

[johnu]

Interesting that you mention thalidomide.

Thalidomide was one of the great success stories of the American food and drug administration FDA.

It was the rigour of the FDA in making sure that it was properly tested that prevented it from entering the US market, which is why the US has no thalidomide victims to speak of.

Maybe there weren't a lot of thalidomide babies in the US, but a bridge playing friend of mine from NY was one of them. He died way too soon.

[johnu]

Why is compliance a problem? Just to give an example, cigarette smoking causes 480,000 deaths every day in the USA. Alcohol causes 95,000 and so on.

Those deaths are per year, not per day. There would be massive depopulation that somebody would notice if they were per day. :lol:

[cherdano]

I'm not a statistics expert, but I think the real point is that developing a COVID-19 vaccine is extremely urgent. Normally it takes several years to develop and test vaccines, this allows you to wait for many results and have much more confidence in the results. But we need something ASAP, we can't wait for thousands of trial participants to die.

 

But there's also more to it. This is Bayesian. We have prior expectations based on the theory behind the vaccine design, and 90 vs 5 is significant enough to confirm the hypoethesis, even though they're both small compared to the group sizes of 15000. It's unlikely to be a statistical fluke since we have good reason to believe that the vaccine will be effective.

That's not how the Pfizer trial works. Their trial was in fact based on a Bayesion model. But the prior expectation they used is that the vaccine will just as likely be <= 30% effective than it will be >= 30%, with a 95% confidence interval ranging from -26.2% to 99.5%. That's conservative, based on the antibody results etc. from the Phase 1/2 trials - I think most experts had a higher prior for efficacy than that.

 

I think many underestimate how drastically these "low numbers" in the trial results change these priors.

 

Let me give an example. Let's say you are 99% convinced that the vaccine is at most 30% effective - in fact, let's be generous (given the trial results) and take that to mean a 99% chance of being _exactly 30%_ effective. You do, however, allow for a 0.99% chance of the vaccine being 80% effective, and a minimal 0.01% of it being 90% effective.

 

After the Pfizer trial, with 8 cases in the vaccine group and 162 in the placebo group, you should change these convictions to there being a

4*10^-20% chance of a 30% effectiveness, a 0.95% chance of it being 80% effective, and a 99.05% chance of it being 90% effective.

 

If efficacy was all there is to it, the trial would have already run waaaay too long. The only reason it wasn't stopped earlier is that the FDA reasonably insisted on two months of safety data after the second shot.

[pilowsky]

Maybe there weren't a lot of thalidomide babies in the US, but a bridge playing friend of mine from NY was one of them. He died way too soon.

 

I'm very sorry to hear that; if it were not for the strength of the regulatory regime and the lack of political interference in it the problem would have been much worse and the suffering much greater.

The current catastrophe in the USA shows what happens when institutions such as the FDA, the CDC and so on are weakened by special interest groups and money is more important than life.

 

 

[pilowsky]

That's not how the Pfizer trial works. Their trial was in fact based on a Bayesion model. But the prior expectation they used is that the vaccine will just as likely be <= 30% effective than it will be >= 30%, with a 95% confidence interval ranging from -26.2% to 99.5%. That's conservative, based on the antibody results etc. from the Phase 1/2 trials - I think most experts had a higher prior for efficacy than that.

 

I think many underestimate how drastically these "low numbers" in the trial results change these priors.

 

Let me give an example. Let's say you are 99% convinced that the vaccine is at most 30% effective - in fact, let's be generous (given the trial results) and take that to mean a 99% chance of being _exactly 30%_ effective. You do, however, allow for a 0.99% chance of the vaccine being 80% effective, and a minimal 0.01% of it being 90% effective.

 

After the Pfizer trial, with 8 cases in the vaccine group and 162 in the placebo group, you should change these convictions to there being a

4*10^-20% chance of a 30% effectiveness, a 0.95% chance of it being 80% effective, and a 99.05% chance of it being 90% effective.

 

If efficacy was all there is to it, the trial would have already run waaaay too long. The only reason it wasn't stopped earlier is that the FDA reasonably insisted on two months of safety data after the second shot.

 

So there are safety trials and efficacy trials, I get that. In this trial 30,000 people were enrolled, 15000 got the active compound and none of them had any real problems. Tick, it's safe.

There were 95 DOCUMENTED cases of infection in the trial, We are not talking about black and white balls in a jar here.

Medicine doesn't work like that. You are probably right - for all our sakes I really want you to be right.

But the numbers are still numbers, and people act and behave differently. It's the same in Bridge.

I have made dozens of antibodies in my lifetime. I've even sold some of them. Images of them have graced the covers of scientific Journals.

Many of them work just fine for one application and then fail dismally for the application that wanted them for.

mRNA constructs are just as bad. I've also worked with RNA and DNA to interfere with biological processes in order to understand physiology.

Some work exactly as they should, some don't.

Everything is great in theory. Later when it comes time to saving lives problems can, and do, arise.

 

When this crisis started I suggested to the local Bridge clubs that it might be a good idea to close. My thoughts were met with derision.

 

You can't fight a virus with statistics.

[cherdano]

The Pfizer trial now has 170 cases, with 162 of them in the placebo group.

 

Obviously the vaccine could be less effective in practice, if the virus mutates or if "real" people are exposed to higher doses than trial participants. But that effect would have to be DRAMATIC to make the vaccine ineffective - not sure such a dramatic shift has ever been observed.

[helene_t]

Helene, one update. In the press release from today, Pfizer/BioNTech specified that it was 8 cases (of symptomatic covid-19) in the vaccine group, and 162 in the placebo group. In the earlier press release from last week, they hadn't given exact numbers, just stated that after 95 cases, the trial indicated an efficacy of > 90%. So they probably had fewer than 8 cases in the vaccine group then.

In that case, I get a 95% CI of 90.6% : 97.8%

[thepossum]

Interesting that the interm (still being reviewed) highest effectiveness reegimen of the Oxford/Astra-Zeneka ChadOx01 trial is also of the order of 90%. The other regimen was 62%, combined so far around 70.4%

 

Oxford Astra Zeneca Interim Announcement

[pilowsky]

The most important part of the Oxford announcement is that the USA vaccines are both only useful in highly urbanised 1st world areas.

I suspect it unlikely that they will get to vaccinate even a third of the US population. That might be an exaggeration, but since a third of the eligible population did not vote and a third voted for Trump, and the vaccine has to be maintained at -ridiculous temperatures,

The chances of rolling it out seem worrying in the real world. Surveys in the USA suggest that many people don't want it even if offered.

 

The Oxford vaccine OTOH was designed to be stored at regular fridge temperatures. I heard the head of the Jenner lab. saying that they wanted to be sure that it could be distributed to all areas of the world. Not just in rich areas.

 

Seriously, what is the point of making a vaccine that only works on a tiny proportion of the population? It's like saying "I'm going to wipe out polio, but only in Texas, and then only if they really want it because we don't want to infringe their civil whatsis".

 

The chap from the Jenner lab (Hill I think) explained that giving the half dose first followed by a boost with a full dose gave 90% efficacy but they were waiting for the US results and more data from the UK - which tragically is coming fast.

 

This variability of the 'priming effect' of immunogens on antibody production doesn't surprise me, we often had to fiddle with the way we gave the immunogens to get a good antibody. The reasons are long and beyond the scope of this Forum.

 

It will be great news if we do get more for less.

[awm]

I think that if I took a vaccine which is 95% effective I would feel pretty safe, whereas if I took a vaccine that was 60-70% effective I wouldn't feel safe until case numbers went down significantly. With a vaccine in the 60-70% range, case numbers will only go down if a lot of people get the vaccine (and/or people are conscientious about taking precautions in the meantime) both of which seem dubious based on current polls and activities.

 

In any case it makes sense to give the expensive and difficult-to-transport (but highly effective) vaccine to health care workers and high-risk people where possible, since they will not be able to wait things out for most people to be vaccinated and case numbers to go down.

 

It also might be a more effective development strategy to create the most effective possible vaccine and then work to bring down the cost, rather than to develop a cheap and easily-portable vaccine and then work to increase the effectiveness.

 

Of course, this is all a bit hypothetical since the percentage effectiveness was computed in different ways for different vaccines so it's not really a fair comparison.

[cherdano]

Interesting that the interm (still being reviewed) highest effectiveness reegimen of the Oxford/Astra-Zeneka ChadOx01 trial is also of the order of 90%. The other regimen was 62%, combined so far around 70.4%

 

Oxford Astra Zeneca Interim Announcement

Last weeks I was here to tell you that the 94% figures or thereabouts by Pfizer and Moderna were as good as they sounded.

This week I am here to tell that that the 90% for the half-dose/full-dose regimen by the Oxford vaccine might not be as good as it sounds.

It's based on a small sample size - just two infections in that group. It's biased due to hypothesis selection - while it's not difficult for experts to come up with a posteriori explanations for the better success of this regimen, they would just as easily have been able to explain if it had been less successful.

 

Still, the 90% could hold up. They also state this regimen leads to fewer PCR-positive participants (regardless of symptoms) - important for reducing transmission. And they had Zero severe cases in either treatment group.

I also wonder whether they could support plausibility of the effectiveness of the half/full regimen by comparing the antibody levels a few weeks after the booster dose.

[pilowsky]

I think that if I took a vaccine which is 95% effective I would feel pretty safe, whereas if I took a vaccine that was 60-70% effective I wouldn't feel safe until case numbers went down significantly. With a vaccine in the 60-70% range, case numbers will only go down if a lot of people get the vaccine (and/or people are conscientious about taking precautions in the meantime) both of which seem dubious based on current polls and activities.

 

In any case it makes sense to give the expensive and difficult-to-transport (but highly effective) vaccine to health care workers and high-risk people where possible, since they will not be able to wait things out for most people to be vaccinated and case numbers to go down.

 

It also might be a more effective development strategy to create the most effective possible vaccine and then work to bring down the cost, rather than to develop a cheap and easily-portable vaccine and then work to increase the effectiveness.

 

Of course, this is all a bit hypothetical since the percentage effectiveness was computed in different ways for different vaccines so it's not really a fair comparison.

 

The 60-70 is false.

They are claiming 90% with the other dosing regime. The media have confused the press release by stating 60-90%.

It's either 60-70% or it's ~90% we don't know yet.

 

The efficacy of the other vaccines in practice depends on the reliability of the cold chain logistics and a lot of other factors. Vaccination in and out of the confines of a clinical trial are two different things.

[johnu]

and the vaccine has to be maintained at -ridiculous temperatures

The Pfizer vaccine has to be kept at -70C, dry ice temperatures, so you need very special refrigeration units to transport and store shipments. The Moderna vaccine is stable at regular freezer temperatures, about -20, and can be kept at refrigerator temperatures for 30 days. The Moderna vaccine seems much better for shipping to remote and rural areas.

 

Neither are as good for shipping as Astra Zeneca, but based on preliminary press releases they have better results. Obviously press releases are not the final word and we'll have to wait for the actual study results to be analyzed to get a better handle on which vaccine is likely to be more effective.

[pilowsky]

Last weeks I was here to tell you that the 94% figures or thereabouts by Pfizer and Moderna were as good as they sounded.

This week I am here to tell that that the 90% for the half-dose/full-dose regimen by the Oxford vaccine might not be as good as it sounds.

It's based on a small sample size - just two infections in that group. It's biased due to hypothesis selection - while it's not difficult for experts to come up with a posteriori explanations for the better success of this regimen, they would just as easily have been able to explain if it had been less successful.

 

Still, the 90% could hold up. They also state this regimen leads to fewer PCR-positive participants (regardless of symptoms) - important for reducing transmission. And they had Zero severe cases in either treatment group.

I also wonder whether they could support plausibility of the effectiveness of the half/full regimen by comparing the antibody levels a few weeks after the booster dose.

 

You are quite right on this latter point. The dosing regimen is key.

The head of the Jenner lab was very cautious. Fortunately, scientists tend not to speculate as wildly as Sydney Powell or Rudy. Except perhaps at grant time...

Nonetheless, he did point out that because of the current resurgence of cases in the UK a better answer is likely to come fairly quickly.

[thepossum]

Last weeks I was here to tell you that the 94% figures or thereabouts by Pfizer and Moderna were as good as they sounded.

This week I am here to tell that that the 90% for the half-dose/full-dose regimen by the Oxford vaccine might not be as good as it sounds.

It's based on a small sample size - just two infections in that group. It's biased due to hypothesis selection - while it's not difficult for experts to come up with a posteriori explanations for the better success of this regimen, they would just as easily have been able to explain if it had been less successful.

 

Still, the 90% could hold up. They also state this regimen leads to fewer PCR-positive participants (regardless of symptoms) - important for reducing transmission. And they had Zero severe cases in either treatment group.

I also wonder whether they could support plausibility of the effectiveness of the half/full regimen by comparing the antibody levels a few weeks after the booster dose.

 

I did say it was up for review and it was only one regimen, and its only interim results, and I only put it up for interest, and I'm sick of all the media beatups and BS. This is serious stuff reduced to a stupid PR marketing game with politcs and all sorts thrown in for the last 10 months. I actuallyput up a serious article for once

 

You have shares in Moderna and Pfizer or something Cherdano? :)

 

What I am actually interested in, is the very small number of cases from such large trials - all of them - and the nature of the analysis and trust in the results. Will be interesting to read all the final studies. And I was also interested that give or take the issues you mention they are all in similar ballparks :)

 

But bearing in mind the point you raise about different groups (and the small numbers) is it reasonable for me to ask a question what it would/could mean in relation to the two regimens and the overall combined result. I appreciate it may not be appropriate to disucss at that level at this stage

[cherdano]

You have shares in Moderna and Pfizer or something Cherdano? :)

No, just waiting for the right time to buy AstraZeneca stocks after their stock has taken enough for a hit :D