[R] inconsistency in mclapply.....

[akshay kulkarni]

Dear Ivan,
                  Thanks for the reply. I am pressurised by a fast approaching deadline and your reply calmed me...

Take a look at the following code:


debug at <tmp>#26: LYG <- mclapply(LYGH, FUN = arfima, mc.cores = 2, mc.preschedule = FALSE)
Browse[2]> length(LYGH)
[1] 357
Browse[2]>
^C

Browse[2]> LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = FALSE)
  |                                                                                                                                                     |   0%, ETA NA

I am debuuging a function FUN wherein the above expressions appear. The pbmclapply code works well if called inside FUN:
> FUN(arg)
Result..

But as you may note, it doesn't work while in debug mode....

Also, if I replace pbmclapply by maclapply inside FUN, it hangs....

You might be interested in this:

[ec2-user using ip-172-31-15-116 ~]$ exit
logout
There are stopped jobs.

THis occurs when I close R and try to exit the shell prompt( I am on an AWS EC2 RHEL 8 Instance). Can this lead you somewhere? As of now I have quit R in my machine, so I can't get session info..but please let me know if you need it necessarily...

by the by, how do you run top when running R? I think at least in my machine, you have to quit R to get to the shell prompt...

I request you TO PLEASE reply to this mail as early as possible. I am facing an imminent deadline...please excuse my blatant violation of protocol, but deadlines are deadlines, right?

THanking you,
Yours sincerely,
AKSHAY M KULKARNI


________________________________
From: Ivan Krylov <krylov.r00t using gmail.com>
Sent: Saturday, June 10, 2023 1:43 AM
To: akshay kulkarni <akshay_e4 using hotmail.com>
Cc: R help Mailing list <r-help using r-project.org>
Subject: Re: [R] inconsistency in mclapply.....

On Fri, 9 Jun 2023 18:01:44 +0000
akshay kulkarni <akshay_e4 using hotmail.com> wrote:

> > LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule =
> > FALSE)
>   |
>                                                                                   |
>   0%, ETA NA^
>
> It just hangs.

My questions from the last time still stand:

0) What is your sessionInfo()? Maybe you're running a parallel BLAS
which doesn't always handle fork() or something. It may be worth
disabling BLAS-level parallelism as long as you're already trying to
use 100% of your processor by other means.

1) What does traceback() show after you interrupt pbmclapply? Most
likely, you would be interrupting selectChildren(), but if not, the
problem may lie in a very different place from what I'm expecting.

2) While pbmclapply is hung, without interrupting it, take a look at
the state of the system and the processes on it (are you still on RHEL?
use `top` or whatever task manager you're comfortable with).

 a) Is 100% of the CPU being used? 100% of one core? Is system mostly
 idle?

 b) Can you find the child processes launched by pbmclapply?

 c) Write down the PID of the child process and attach a debugger to
 it (If you're on RHEL, try following this guide:
 <https://beej.us/guide/bggdb/#attach>. If GDB asks you to install
 additional debug symbols by running debuginfo-install, follow its
 guidance and then restart GDB.) and obtain a backtrace. (In GDB, the
 command to obtain a backtrace is "backtrace".) Which function is the
 child process stuck in?

--
Best regards,
Ivan

	[[alternative HTML version deleted]]