[R] Use of R in clinical trials

[Frank E Harrell Jr]

Bert,

I have to disagree with just part of what you said.  The ultimate 
savings by using R is astronomical.  Up front it would definitely cost 
more, as you so eloquently stated.  So it boils down to short-term vs. 
long-term thinking.

More importantly, the statistical/graphical reports created using R are 
more informative and lead to better reviews and more clear assessment of 
pharmaceutical safety problems by not relying on reams of tables that 
put reviewers to sleep.

Frank


Bert Gunter wrote:
> DISCLAIMER: This represents my personal view and in no way reflects that of
> my company.
> 
> Warning: This is a long harangue that contains no useful information on R.
> May be wise to delete without reading. 
> ----------
> 
> Sorry folks, I still don't understand your comments. As Cody's original post
> pointed out, there are a host of factors other than ease of programmability
> or even quality of results that weigh against any change. To reiterate, all
> companies have a huge infrastructure of **validated SAS code** that would
> have to be replaced. This, in itself, would take years and cost tens of
> millions of dollars at least. Also to reiterate, it's not only
> statistical/reporting functionality but even more the integration into the
> existing clinical database systems that would have to be rewritten **and
> validated**. All this would have to be done while continuing full steam on
> existing submissions. It is therefore not surprising to me that no pharma
> company in its right mind even contemplates undertaking such an effort.
> 
> To put these things into perspective. Let's say Pfizer has 200 SAS
> programmers (it's probably more, as they are a large Pharma, but I dunno).
> If each programmer costs, conservatively, $200K U.S. per year fully loaded,
> that's $40 million U.S. for SAS Programmers. And this is probably a severe
> underestimate. So the $14M quoted below is chicken feed -- it doesn't even
> make the radar. 
> 
> To add further perspective, a single (large) pivotal clinical trial can
> easily cost $250M . A delay in approval due to fooling around trying to
> shift to a whole new software system could easily cause hundreds of million
> to billions if it means a competitor gets to the marketplace first. So, to
> repeat, SAS costs are chicken feed.
> 
> Yes, I suppose that the present system institutionalizes mediocrity. How
> could it be otherwise in any such large scale enterprise? Continuity,
> reliability, and robustness are all orders of magnitude more important for
> both the FDA and Pharma to get safe and efficacious drugs to the public.
> Constantly hopping onto the latest and greatest "craze" (yes, I exaggerate
> here!) would be dangerous, unacceptable, and would probably delay drug
> approvals. I consider this another example of the Kuhnsian paradigm (Thomas
> Kuhn: "The Structure of Scientific Revolutions")in action.
> 
> This is **not** to say that there is not a useful role for R (or STATA or
> ...) to play in clinical trial submissions or, more generally, in drug
> research and development. There certainly is. For the record, I use R
> exclusively in my (nonclinical statistics) work. Nor is to say that all
> change must be avoided. That would be downright dangerous. But let's please
> keep these issues in perspective. One's enthusiasm for R's manifold virtues
> should not replace common sense and logic. That, too, would be unfortunate.
> 
> Since I've freely blustered, I am now a fair target. So I welcome forceful
> rebuttals and criticisms and, as I've said what I wanted to, I will not
> respond. You have the last word.
> 
> Bert Gunter
> Genentech Nonclinical Biostatistics
>  
> 
> 
>  -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
> Behalf Of Frank E Harrell Jr
> Sent: Thursday, February 18, 2010 6:01 AM
> To: Bill.Venables at csiro.au
> Cc: r-help at r-project.org; p.dalgaard at biostat.ku.dk
> Subject: Re: [R] Use of R in clinical trials
> 
> I really like both of your responses.  To add to Peter's thoughts, I've 
> found that more than half of SAS programmers can learn modern 
> programming languages given a push.  And if pharmaceutical companies 
> ever knew the true cost of SAS in terms of their having to hire more 
> programmers to deal with an archaic language they would be astonished. 
> Rumor had it that Pfizer's yearly SAS licensing costs were $14M/year 
> several years ago.  Programmer costs were probably in the same range.
> 
> Frank
> 
> 
> Bill.Venables at csiro.au wrote:
>> I can't believe I'm saying this, but I think Peter is being a bit harsh on
> SAS.  
>> I prefer Greg Snow's analogy (in the fortune collection): If SPSS (or SAS)
> and R were vehicles, SPSS would be the bus, going on fixed routes and
> efficiently carrying lots of people to standard places, whereas R is the
> off-road 4WD SUV, complete with all sorts of kit including walking boots,
> kayak on the top, &c.  R will take you anywhere you want to go, but it might
> take you longer to master it than the simple recipes for data analysis
> typical of the 'bus' programs.
>>
>> Bill Venables
>> CSIRO/CMIS Cleveland Laboratories
>>
>>
>> -----Original Message-----
>> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
> On Behalf Of Peter Dalgaard
>> Sent: Thursday, 18 February 2010 5:55 PM
>> To: Frank E Harrell Jr
>> Cc: r-help at r-project.org; Cody Hamilton
>> Subject: Re: [R] Use of R in clinical trials
>>
>> Frank E Harrell Jr wrote:
>>> Cody,
>>>
>>> How amazing that SAS is still used to produce reports that reviewers 
>>> hate and that requires tedious low-level programming.  R + LaTeX has it 
>>> all over that approach IMHO.  We have used that combination very 
>>> successfully for several data and safety monitoring reporting tasks for 
>>> clinical trials for the pharmaceutical industry.
>>>
>>> Frank
>> There is a point to it, though. One of my friends and colleagues in the 
>> business put it in one word: Mediocrity.
>>
>> SAS does a mediocre job at analysing and reporting and data handling 
>> using a mediocre control language. But: It can be handled by mediocre 
>> programmers writing and modifying mediocre programs, and those people 
>> are more available and replaceable, maybe even cheaper. R/LaTeX may run 
>> circles around SAS in terms of capapilities, flexibility, and elegance, 
>> but it can also send a programmer who doesn't have the required skill 
>> set running around in circles.
>>
>> -pd
>>
>>> Cody Hamilton wrote:
>>>> Dear all,
>>>>
>>>> There have been a variety of discussions on the R list regarding the 
>>>> use of R in clinical trials. The following post from the STATA list 
>>>> provides an interesting opinion regarding why SAS remains so popular 
>>>> in this arena: 
>>>> http://www.stata.com/statalist/archive/2008-01/msg00098.html
>>>>
>>>> Regards,
>>>>
>>>> -Cody Hamilton
>>
> 
> 


-- 
Frank E Harrell Jr   Professor and Chairman        School of Medicine
                      Department of Biostatistics   Vanderbilt University