Lwt and exceptions

[r***@happyleptic.org]

The Lwt doc states that you should not use "raise" when using Lwt
but use Lwt.fail instead.

So, is it still OK to call functions (for instance from the stdlib)
that may raise an exception, provided we catch it soon enough ?
And by "soon enough" I mean: before an Lwt call that could schedule
another thread.

[Anil Madhavapeddy]

Yes, it's essential to do this for many common functions such as Hashtbl.find. The nice thing about Lwt is that you control yielding, so as long as you catch the exception locally and "convert" it into the Lwt monad, everything works great.

The occasional exception leak is really hard to track down though, if you let one propagate by mistake. Ban OCaml exceptions! :)

Anil

Post by r***@happyleptic.org
The Lwt doc states that you should not use "raise" when using Lwt
but use Lwt.fail instead.
So, is it still OK to call functions (for instance from the stdlib)
that may raise an exception, provided we catch it soon enough ?
And by "soon enough" I mean: before an Lwt call that could schedule
another thread.
--
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[r***@happyleptic.org]

-[ Tue, Sep 13, 2011 at 08:17:24PM +0100, Anil Madhavapeddy ]----

Post by Anil Madhavapeddy
The occasional exception leak is really hard to track down though, if you let one propagate by mistake. Ban OCaml exceptions! :)

Yes. Also, I'd like a Lwt.assert syntax extension to replace the
assert keyword. :-(

[Jérémie Dimino]

Post by r***@happyleptic.org
-[ Tue, Sep 13, 2011 at 08:17:24PM +0100, Anil Madhavapeddy ]----

Post by Anil Madhavapeddy
The occasional exception leak is really hard to track down though, if you let one propagate by mistake. Ban OCaml exceptions! :)

Yes. Also, I'd like a Lwt.assert syntax extension to replace the
assert keyword. :-(

I have added the assert_lwt keyword in pa_lwt.
--
Jérémie

[Mehdi Dogguy]

The Lwt doc states that you should not use "raise" when using Lwt but
use Lwt.fail instead.
So, is it still OK to call functions (for instance from the stdlib)
that may raise an exception, provided we catch it soon enough ? And by
"soon enough" I mean: before an Lwt call that could schedule another
thread.

I guess, not (and it has been answered already). In fact, I was wondering
if Lwt's authors would be against adding a function like:

let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e

It is stupid, trivial, etc… but looks what we need most of the time, no?
Instead of doing it in our own code, it could land in Lwt directly.
But, if it gets integrated into Lwt proper, users should be warned about
its behaviour. (especially with impure functions).

Regards,

[Anil Madhavapeddy]

Post by Mehdi Dogguy

The Lwt doc states that you should not use "raise" when using Lwt but
use Lwt.fail instead.
So, is it still OK to call functions (for instance from the stdlib)
that may raise an exception, provided we catch it soon enough ? And by
"soon enough" I mean: before an Lwt call that could schedule another
thread.

I guess, not (and it has been answered already). In fact, I was wondering
let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e
It is stupid, trivial, etc… but looks what we need most of the time, no?
Instead of doing it in our own code, it could land in Lwt directly.
But, if it gets integrated into Lwt proper, users should be warned about
its behaviour. (especially with impure functions).

The try_lwt construct (in pa_lwt) or try_bind already converts normal exceptions into Lwt ones:

try_lwt
raise (Failure "")
with Failure _ ->
print_endline "Fail"; Lwt.return ()

...will print "Fail".

You just need to careful about raising exceptions across yield points when a try_lwt isn't present. The slightly painful thing about converting existing code to Lwt is that any function that raises an exception will gain the Lwt type if you convert it to use try_lwt/catch, even if the rest of the code doesn't otherwise block.

Anil

[Jérémie Dimino]

Post by Mehdi Dogguy
I guess, not (and it has been answered already). In fact, I was wondering
let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e
It is stupid, trivial, etc… but looks what we need most of the time, no?

Yes, it seems useful. But should it be:

val wrap : ('a -> 'b) -> 'a -> 'b t

or:

val wrap : (unit -> 'a) -> 'a t

?

I would tend for the second solution because if you are wrapping a
function that takes multiple arguments you are going to write:

wrap (fun () -> f x y z) ()

anyway. Plus maybe wrap1, wrap2, ..., wrapn for a reasonable value of n.
--
Jérémie

[Mehdi Dogguy]

Post by Jérémie Dimino

Post by Mehdi Dogguy
I guess, not (and it has been answered already). In fact, I was wondering
let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e
It is stupid, trivial, etc… but looks what we need most of the time, no?

val wrap : ('a -> 'b) -> 'a -> 'b t
val wrap : (unit -> 'a) -> 'a t
?
I would tend for the second solution because if you are wrapping a
wrap (fun () -> f x y z) ()
anyway. Plus maybe wrap1, wrap2, ..., wrapn for a reasonable value of n.

Sure. Thanks for considering. Any of the above proposals would be fine for
me :)

[Dmitry Grebeniuk]

Hello.

Post by Mehdi Dogguy
In fact, I was wondering
       let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e

Not a very good idea, since evaluation of wrap'ped
function's arguments can raise an exception that
won't be caught by wrap, suppose:
let lwt_string_of_int = wrap string_of_int
let () = Lwt_main.run (
lwt_string_of_int (raise Exit) >>= fun s ->
Lwt_io.printf "everything's okay buddies, it's %s\n" s
)

So it's up to the caller to guarantee the absence of
native exceptions. It's bad. If I will ever need to write
such wrapper, I will make it to have type
wrap : (unit -> ('a -> 'b)) -> (unit -> 'a) -> 'b Lwt.t
maybe with the name of function too, and with
early evaluation of first argument, like
let wrap ?(funcname="") u_f =
let r_f =
try
`Ok (u_f ())
with e_f ->
`Error e_f
in
match r_f with
| `Ok f -> fun x ->
try
Lwt.return (f x)
with e_x ->
(* 1.either *)
Lwt.fail (Failure (Printf.sprintf
"error evaluating function %S: %s"
funcname (Printexc.to_string e_x)))
(* 1.or *)
Lwt.fail e_x
| `Error e_f ->
(* 2.either *)
failwith (Printf.sprintf
"the wrapped function %S itself \
raises error: %s"
funcname (Printexc.to_string e_f))
(* 2.or *)
fun _u_x ->
Lwt.fail e_f

(I haven't checked this code! I don't remember
where should I set begin-end or parenthesis in
nested matches in the original syntax, and I don't
want to remember it now.)

I like the uniform error handling. So I have
wrapped the common IO-specific functions
to a library that gives IO_lwt for lwt, IO_direct
for direct OCaml I/O (stdlib), and... functor
functor functor it sometimes.
It uses "bind" and "return", but I'm very unsure
whether the code can be called "monadic IO",
since lwt itself does not respect the monad laws.
But it doesn't matter, since I'm not a academician,
I'm an engineer.
So I'm using
IO.catch : (unit -> 'a m) -> (exn -> 'a m) -> 'a m
even for direct IO computations.

[Jérémie Dimino]

Post by Dmitry Grebeniuk
It uses "bind" and "return", but I'm very unsure
whether the code can be called "monadic IO",
since lwt itself does not respect the monad laws.

What monadic laws lwt does not respect ?
--
Jérémie

[Dmitry Grebeniuk]

Hello.

Post by Jérémie Dimino

Post by Dmitry Grebeniuk
It uses "bind" and "return", but I'm very unsure
whether the code can be called "monadic IO",
since lwt itself does not respect the monad laws.

What monadic laws lwt does not respect ?

Lwt sees values of type Lwt.t not as IO actions,
but sometimes as the values of these actions.

It does not respect the monad associativity:

(m >>= f) >>= g == m >>= (fun x -> f x >>= g)

Look at the source, see the left and the right
parts of this law as the "res" values, uncomment
either block, run the program, enter two lines
of text to program's stdin and see the output.
The output should be the same in both cases
if the law is respected.

==============
open Lwt;

value () = Printf.printf "case 1, enter two lines:\n%!";
value m = Lwt_io.read_line Lwt_io.stdin;
value f_v = Lwt_io.read_line Lwt_io.stdin;
value f = fun _ -> f_v;
value g = Lwt.return;
value res = (m >>= f) >>= g;

(*
value () = Printf.printf "case 2, enter two lines:\n%!";
value f_v = Lwt_io.read_line Lwt_io.stdin;
value m = Lwt_io.read_line Lwt_io.stdin;
value f = fun _ -> f_v;
value g = Lwt.return;
value res = m >>= (fun x -> f x >>= g);
*)

value () = Lwt_main.run (res >>= fun s ->
Lwt_io.write_line Lwt_io.stdout ("res = " ^ s));
==============

In the first case it's:

case 1, enter two lines:
a
b
res = b

In the second case it's:

case 2, enter two lines:
a
b
res = a

When I began to use Lwt, I was badly surprised
by the fact that Lwt.t values are not real I/O actions.
So I had to get a new habit: if the top-level module
value has type Lwt.t 'a, I have to defer its
computation by wrapping it under "fun () -> <expr>".
And I have to remember which of my functions return
Lwt.t values, which of my abstract types really have
type Lwt.t 'a or contain the Lwt.t 'a values, to defer
their computations/sideeffects too.

[Stéphane Glondu]

Post by Dmitry Grebeniuk
(m >>= f) >>= g == m >>= (fun x -> f x >>= g)
[...]
==============
open Lwt;
value () = Printf.printf "case 1, enter two lines:\n%!";
value m = Lwt_io.read_line Lwt_io.stdin;
value f_v = Lwt_io.read_line Lwt_io.stdin;
value f = fun _ -> f_v;
value g = Lwt.return;
value res = (m >>= f) >>= g;
(*
value () = Printf.printf "case 2, enter two lines:\n%!";
value f_v = Lwt_io.read_line Lwt_io.stdin;
value m = Lwt_io.read_line Lwt_io.stdin;
value f = fun _ -> f_v;
value g = Lwt.return;
value res = m >>= (fun x -> f x >>= g);
*)
value () = Lwt_main.run (res >>= fun s ->
Lwt_io.write_line Lwt_io.stdout ("res = " ^ s));
==============

Using Lwt doesn't automatigally make OCaml purely functional... in your
example, you change the order of m and f_v, which both do side-effects,
so they are different in the two instances of res, so they cannot be
compared. Can you give an example that involves only pure functions?

Post by Dmitry Grebeniuk
When I began to use Lwt, I was badly surprised
by the fact that Lwt.t values are not real I/O actions.
So I had to get a new habit: if the top-level module
value has type Lwt.t 'a, I have to defer its
computation by wrapping it under "fun () -> <expr>".
And I have to remember which of my functions return
Lwt.t values, which of my abstract types really have
type Lwt.t 'a or contain the Lwt.t 'a values, to defer
their computations/sideeffects too.

Maybe what you expected was something that encapsulates the IO monad
too. Lwt doesn't do that: as with raw OCaml, IOs are direct with Lwt.


Cheers,
--
Stéphane

[Jérémie Dimino]

Post by Dmitry Grebeniuk
When I began to use Lwt, I was badly surprised
by the fact that Lwt.t values are not real I/O actions.
So I had to get a new habit: if the top-level module
value has type Lwt.t 'a, I have to defer its
computation by wrapping it under "fun () -> <expr>".
And I have to remember which of my functions return
Lwt.t values, which of my abstract types really have
type Lwt.t 'a or contain the Lwt.t 'a values, to defer
their computations/sideeffects too.

You are comparing Lwt with the IO monad. They are two different monads:
IO deals with actions while Lwt deals with threads. If you write:

let m = IO.read_char IO.stdin

then [m] is just the description of the action of reading a character
from [stdin]. On the contrary, if you write:

let m = Lwt_io.read_char Lwt_io.stdin

then [m] is really a thread waiting for a character from [stdin]. If you
use "fun () -> ..." everywhere then this becomes actions but it is not
threads anymore and you loose most of the advantages of Lwt. For example
with Lwt you can write:

let t1 = f1 ()
and t2 = f2 ()
...
and tn = fn () in
Lwt.bind t1 (fun x1 -> Lwt.bind t2 (fun x2 -> ... Lwt.bind tn (fun xn -> return (x1, x2, ..., xn)) ...))

and this will let [t1], ..., [tn] run concurrently. With the IO monad
this is just exactly the same as:

let x1 = f1 () in
let x2 = f2 () in
...
let xn = fn () in
(x1, x2, ..., xn)

in a non-monadic world. There is no concurrency at all.

Cheers,
--
Jérémie