Discussion:
Re: Multiple mounts for a single FUSE process.
(too old to reply)
Bill Cox
2005-07-06 10:15:04 UTC
Permalink
Hi.

I'm trying to get basic FUSE support integrated into my BitTorrent
client. I need the multiple mount capability. I also need to be able
to process commands from shell input, and dynamically mount and unmount
based on those commands. My BitTorrent client supports multiple
simultaneous torrent downloads. This is very important, since users
typically want to control total bandwidth allocated to BitTorrent, not
bandwidth per file download in progress. Each torrent can be mounted by
the user from within the command shell.

Like Terje, I was able to get a select-based command processing loop
done in a couple of hours. The do_read, however, still needs to be a
separate thread to make this work. I've been trying for a day now, and
after reading Terje's posts, I see I still have a lot of work do to.

Has there been any progress on official support in FUSE for multiple
mounts? I don't particularly care if the support is pure select-based,
or 100% threads, or a mix. In any case, the do_read operation has to be
suspended while I go download the data (and continue to process shell
commands, other downloads, etc), and when it's done, I signal it to
continue.

Thanks,
Bill
Terje Oseberg
2005-07-07 20:51:25 UTC
Permalink
Have you had a chance to download and take a look at my code:

http://www.unixprogram.com/FUSE1.3/multiplemounts/
&
http://www.unixprogram.com/FUSE1.3/multiplemounts/fuse_10-08-04_15.38.53_experimental.tar.gz

There are 3 versions in there using poll(), select(), and threads to
create 10 mounts with a single process.

It's been a while since I've looked at it, but it seems that it
wouldn't be too difficult to clean it up and make it work with the
latest version of FUSE, then add the ability to add and remove mounts
on the fly with some sort of interface.

However I haven't taken a look at FUSE for several months, and it
seems to be evolving quickly, so I don't have a clue how compatible my
FUSE 1.3 usage is with the latest version.

Terje
Post by Bill Cox
Hi.
I'm trying to get basic FUSE support integrated into my BitTorrent
client. I need the multiple mount capability. I also need to be able
to process commands from shell input, and dynamically mount and unmount
based on those commands. My BitTorrent client supports multiple
simultaneous torrent downloads. This is very important, since users
typically want to control total bandwidth allocated to BitTorrent, not
bandwidth per file download in progress. Each torrent can be mounted by
the user from within the command shell.
Like Terje, I was able to get a select-based command processing loop
done in a couple of hours. The do_read, however, still needs to be a
separate thread to make this work. I've been trying for a day now, and
after reading Terje's posts, I see I still have a lot of work do to.
Has there been any progress on official support in FUSE for multiple
mounts? I don't particularly care if the support is pure select-based,
or 100% threads, or a mix. In any case, the do_read operation has to be
suspended while I go download the data (and continue to process shell
commands, other downloads, etc), and when it's done, I signal it to
continue.
Thanks,
Bill
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Bill Cox
2005-07-07 21:23:20 UTC
Permalink
Hi, Terje.

I did look through your documentation and code. It looks like nice
work. I was able to use your select method with the latest FUSE release
without modifying any FUSE source. However, read commands still hang up
the system until the read request is satisfied, and adding the thread
code for reads seems to require code changes internal to FUSE.

As you've said, FUSE is changing quickly, so doing my own local version
will cause compatibility headaches. Is there any way I can help in an
effort to add multiple mount support to FUSE?

Thanks,
Bill
Post by Terje Oseberg
http://www.unixprogram.com/FUSE1.3/multiplemounts/
&
http://www.unixprogram.com/FUSE1.3/multiplemounts/fuse_10-08-04_15.38.53_experimental.tar.gz
There are 3 versions in there using poll(), select(), and threads to
create 10 mounts with a single process.
It's been a while since I've looked at it, but it seems that it
wouldn't be too difficult to clean it up and make it work with the
latest version of FUSE, then add the ability to add and remove mounts
on the fly with some sort of interface.
However I haven't taken a look at FUSE for several months, and it
seems to be evolving quickly, so I don't have a clue how compatible my
FUSE 1.3 usage is with the latest version.
Terje
Terje Oseberg
2005-07-07 22:47:28 UTC
Permalink
I'm not sure what you mean by, "However, read commands still hang up
the system until the read request is satisfied". It should take 10
reads to cause further commands to hang. This number can be increased
(it's defined somewhere in the code).

I just took a look at the code, and it appears that there is a single
threaded and multi-threaded option for the select and poll versions.
For both, in the single threaded version, a single read can cause the
whole thing to hang until the read returns. Make sure you use the
multi-threaded option. Then it will take 10 reads to hang to cause
everything to hang until one completes (This number is #defined
somewhere as FUSE_MAX_WORKERS)

Regarding getting official support for this version from FUSE, I
believe that the interface used in my code should be officially
supported, but my examples may not be. So if you are able to get my
code to work reliably with the latest version of FUSE without
modifying the FUSE library, you should be able to keep using the newer
and newer versions as they are released, then complain the instant
something breaks, and I'm sure the FUSE guys will accommodate you.
(Miklos, any comments?)

Terje
Post by Bill Cox
Hi, Terje.
I did look through your documentation and code. It looks like nice
work. I was able to use your select method with the latest FUSE release
without modifying any FUSE source. However, read commands still hang up
the system until the read request is satisfied, and adding the thread
code for reads seems to require code changes internal to FUSE.
As you've said, FUSE is changing quickly, so doing my own local version
will cause compatibility headaches. Is there any way I can help in an
effort to add multiple mount support to FUSE?
Thanks,
Bill
Post by Terje Oseberg
http://www.unixprogram.com/FUSE1.3/multiplemounts/
&
http://www.unixprogram.com/FUSE1.3/multiplemounts/fuse_10-08-04_15.38.53_experimental.tar.gz
There are 3 versions in there using poll(), select(), and threads to
create 10 mounts with a single process.
It's been a while since I've looked at it, but it seems that it
wouldn't be too difficult to clean it up and make it work with the
latest version of FUSE, then add the ability to add and remove mounts
on the fly with some sort of interface.
However I haven't taken a look at FUSE for several months, and it
seems to be evolving quickly, so I don't have a clue how compatible my
FUSE 1.3 usage is with the latest version.
Terje
Miklos Szeredi
2005-07-08 08:46:21 UTC
Permalink
Post by Terje Oseberg
Regarding getting official support for this version from FUSE, I
believe that the interface used in my code should be officially
supported, but my examples may not be. So if you are able to get my
code to work reliably with the latest version of FUSE without
modifying the FUSE library, you should be able to keep using the newer
and newer versions as they are released, then complain the instant
something breaks, and I'm sure the FUSE guys will accommodate you.
(Miklos, any comments?)
I would be happy to integrate multi-mount support, if it's done nicely
enough. I'd prefer if it wasn't a separate piece of code, rather
integrated into the single-mount code.

Miklos
Bill Cox
2005-07-08 11:32:04 UTC
Permalink
Post by Miklos Szeredi
Post by Terje Oseberg
Regarding getting official support for this version from FUSE, I
believe that the interface used in my code should be officially
supported, but my examples may not be. So if you are able to get my
code to work reliably with the latest version of FUSE without
modifying the FUSE library, you should be able to keep using the newer
and newer versions as they are released, then complain the instant
something breaks, and I'm sure the FUSE guys will accommodate you.
(Miklos, any comments?)
I would be happy to integrate multi-mount support, if it's done nicely
enough. I'd prefer if it wasn't a separate piece of code, rather
integrated into the single-mount code.
Miklos
Cool. I'd be happy to help prototype any changes, as Terje has done so
far. It sounds to me from reading the latest posts that the approach to
use still had not been settled.

IMO, the work done so far is fine, and if the example interface from
either the threads example or select example were added, it'd solve my
problem.

However, I share Miklos's concern about whether we've got the best
approach, yet. In particular, wouldn't it be a smaller change if we had
the user launch fuse_main from a separate thread for each mount? There
is very little use of globals in the fuse code. Would it be hard to
make it 100% thread-safe? The nice thing there is that the interface
wouldn't need to change at all. All we'd add is a mutex that the user
can lock when he wants all the fuse threads to stop mucking with his
data structures. The way I'd use it would be to lock the mutex in my
main program all the time, but unlock it before I call my main select
function, and lock it again right after.

Thanks,
Bill
Vincenzo Ciancia
2005-07-08 11:46:14 UTC
Permalink
Post by Bill Cox
I did look through your documentation and code. It looks like nice
work. I was able to use your select method with the latest FUSE release
without modifying any FUSE source. However, read commands still hang up
the system until the read request is satisfied, and adding the thread
code for reads seems to require code changes internal to FUSE.
And I would like to remind that if it's possible it would be preferable to
use asynchronous message passing instead of C threads, to allow usage from
ocaml, c# and other programming languages which do not allow callbacks from
different C threads.

Bye and thanks for working for all us :)

Vincenzo
--
Please note that I do not read the e-mail address used in the from field but
I read vincenzo_ml at yahoo dot it
Attenzione: non leggo l'indirizzo di posta usato nel campo from, ma leggo
vincenzo_ml at yahoo dot it
Bill Cox
2005-07-08 18:04:51 UTC
Permalink
I think Milkos prefers threads, while Terje seems
to prefer the select based version without threads. In my experience,
it's what the project maintainer likes that counts, so I suspect we'll
go with threads.
But, I repeat, it's not a matter of taste, it's a matter of binding to
languages different than C (BTW I am the developer and mantainer of the
ocaml binding) - however I guess the best thing to do is to discuss this
particular issue on the fuse mailing list, and hear the opinions of the
various developers, including those developing other language bindings.
Very cool! I'm a big fan of ocaml. Also, that makes you the right guy
for this discussion :-)
And primarily, I want to underline, I said this to you just to make sure that
you knew the cons, besides the pros, of a threaded approach. If you develop
it's you who chose how to develop.
I agree. So, will threads cause problem with an ocaml binding? I don't
think it needs call-backs.
Valent Gough (if I don't mispell his name) had to rewrite libfuse to bind fuse
to C#, he succeded but he won't benefit of libfuse improvements (like
multiple mounts). He might as well shed more light on the subject, and there
also is some discussion about that in the mailing list archives.
Good idea.
I am leaving for two weeks to attend a summer school so I guess I won't take
part in the discussion next week, but maybe I will if there is internet
connection there.
I will be out of town all next week as well :-)
Valient Gough
2005-07-09 15:03:37 UTC
Permalink
Post by Bill Cox
Valent Gough (if I don't mispell his name) had to rewrite libfuse to bind
fuse to C#, he succeded but he won't benefit of libfuse improvements
(like multiple mounts). He might as well shed more light on the subject,
and there also is some discussion about that in the mailing list
archives.
Good idea.
Well I just returned from being gone all week. I gave a talk on Encfs
(libfuse based) at the LSM conference in Dijon. Part of the talk went over
FUSE and libfuse, so hopefully I didn't misrepresent it :-)

The problem interfacing with C# (and some other languages, although I don't
know ocaml enough to talk about it), is that the language is not tolerant of
thread it didn't create. I believe this may be normal for any language which
uses a garbage collector. In C#, the garbage collector can run any time and
can move things around in memory, etc. So if there are threads it doesn't
know about, they can be accessing data while it is being moved. It isn't
just theoretical, my first attempt at a C# wrapper for libfuse would die very
quickly in threaded mode.

Mono's C# embedded interface does have a way to allow registering threads. I
tried using a thread-local variable to track threads and register new ones,
but that was difficult and never worked. I could have exposed a thread
creation method from libfuse, however that still would have tied me to the
Mono internal interface. Instead I wrote an interface in C#. It still uses
a couple functions from libfuse - such as startup functions to create the
mount.

My C# layer already supports multiple mounts in a process, which was pretty
trivial over supporting just one. The design is that a Channel object exists
for every connection to the kernel (each mount point). That handles basic
kernel communication and provides asynchronous callbacks when a message is
received. Then a Reactor class handles the decoding of the message and calls
into the appropriate node. There can either be one Reactor serving all
Channel objects, or one Reactor per Channel.

The easiest C interface I can think of for interfacing to a language binding
would be an asynchronous interface without creating threads. That would
allow the language binding to create threads as appropriate in the language.
For example, when libfuse calls a callback, then the handler (the language
binding) could create a thread to handle the function and then immediately
return to libfuse for further event processing. Then once the callback
finished in the other thread, the response could be sent via a separate
function call.

regards,
Valient
Terje Oseberg
2005-07-11 21:09:03 UTC
Permalink
Post by Valient Gough
Post by Bill Cox
Valent Gough (if I don't mispell his name) had to rewrite libfuse to bind
fuse to C#, he succeded but he won't benefit of libfuse improvements
(like multiple mounts). He might as well shed more light on the subject,
and there also is some discussion about that in the mailing list
archives.
Good idea.
.
.
.
Post by Valient Gough
The easiest C interface I can think of for interfacing to a language binding
would be an asynchronous interface without creating threads. That would
allow the language binding to create threads as appropriate in the language.
For example, when libfuse calls a callback, then the handler (the language
binding) could create a thread to handle the function and then immediately
return to libfuse for further event processing. Then once the callback
finished in the other thread, the response could be sent via a separate
function call.
regards,
Valient
And this is why I created the prototype Asynchronous Fuse library.
Although the library is a bit convoluted, and could be refactored to
be much simpler, it works, and I believe that the interface is mostly
easy. I think that a better implementation should use a very similar
interface.

http://www.unixprogram.com/FUSE1.9asynchronous/

Terje
Valient Gough
2005-07-11 21:49:44 UTC
Permalink
Post by Terje Oseberg
Post by Valient Gough
The easiest C interface I can think of for interfacing to a language
binding would be an asynchronous interface without creating threads.
That would allow the language binding to create threads as appropriate in
the language. For example, when libfuse calls a callback, then the
handler (the language binding) could create a thread to handle the
function and then immediately return to libfuse for further event
processing. Then once the callback finished in the other thread, the
response could be sent via a separate function call.
And this is why I created the prototype Asynchronous Fuse library.
Although the library is a bit convoluted, and could be refactored to
be much simpler, it works, and I believe that the interface is mostly
easy. I think that a better implementation should use a very similar
interface.
http://www.unixprogram.com/FUSE1.9asynchronous/
I remember you talking about this a while back, but I'm afraid I never really
looked closely at your work, although I like the sound of it.

But the big question is how well will it be supported? Any replacement will
have to be something that Miklos will support (say by being integrated into
libfuse), otherwise it will become stale without active maintenance..

From what I gather, Miklos would [quite reasonably] prefer to see a set of
incremental patches rather then a big code dump (not unlike say the linux
kernel developers).

So, I really should look more at your library in order to be able to ask
intelligent questions. Can we take what you learned in creating the
interfaces and start proposing small changes which add support for
asynchronous calls to some of the methods, or did you basically end up with a
rewrite?

I read the comments on your web page. Why doesn't it work in single-threaded
mode? I think that is the mode we should be concentrating on anyway, since
it would allow the client to handle threading (useful for language bindings).

Valient
Terje Oseberg
2005-07-11 22:38:31 UTC
Permalink
Post by Valient Gough
Post by Terje Oseberg
Post by Valient Gough
The easiest C interface I can think of for interfacing to a language
binding would be an asynchronous interface without creating threads.
That would allow the language binding to create threads as appropriate in
the language. For example, when libfuse calls a callback, then the
handler (the language binding) could create a thread to handle the
function and then immediately return to libfuse for further event
processing. Then once the callback finished in the other thread, the
response could be sent via a separate function call.
And this is why I created the prototype Asynchronous Fuse library.
Although the library is a bit convoluted, and could be refactored to
be much simpler, it works, and I believe that the interface is mostly
easy. I think that a better implementation should use a very similar
interface.
http://www.unixprogram.com/FUSE1.9asynchronous/
I remember you talking about this a while back, but I'm afraid I never really
looked closely at your work, although I like the sound of it.
But the big question is how well will it be supported? Any replacement will
have to be something that Miklos will support (say by being integrated into
libfuse), otherwise it will become stale without active maintenance..
Definatly! My code is already stale less than 6 months from it's creation.
Miklos Szeredi
2005-07-12 13:52:34 UTC
Permalink
Our hopes were that Miklos would adopt the interface in his
implementation to allow what we write on top of the asynchronous
interface to be compatible with whatever Miklos comes up with.
I think the interface was basically OK (and still is, probably). I
didn't merge it because the implementation was not clean enough for my
sensitive taste ;)

I've started implementing the inode based API, which will be async
only (CVS already has non-working bits and pieces). It should be
easier to implement the async path based API on top of this.

Miklos
Terje Oseberg
2005-07-12 20:50:50 UTC
Permalink
Post by Miklos Szeredi
Our hopes were that Miklos would adopt the interface in his
implementation to allow what we write on top of the asynchronous
interface to be compatible with whatever Miklos comes up with.
I think the interface was basically OK (and still is, probably). I
didn't merge it because the implementation was not clean enough for my
sensitive taste ;)
I've started implementing the inode based API, which will be async
only (CVS already has non-working bits and pieces). It should be
easier to implement the async path based API on top of this.
I agree ("It should be easier to implement the async path based API on
top of this"). And this is the correct way to do it. I just was on a
deadline and needed to get this done quick, so I did quick and dirty.
I just made sure that the interface was good so that it could be
something that could be kept. If possible it'd be nice to have the new
asynchronous library that's build on top of the inode based library
work with the examples that I provided with my version of the
asynchronous library with minimal changes.

Terje

Continue reading on narkive:
Loading...