Post by Michael McTernanPost by Stef Bonyou can do anything with FUSE, but isn't the problem here the decoder
using a lock, cause of reasons you've descibed. It cannot hanlde fifo
queueing, what causes this.
If the filesystem was lockless, then yes there would be no problem.
However, it's a really really common case that filesystems need to
serialise access to each specific file, and hence they will use a lock
(or opt of single threaded).
Please explain here a litlle bit about lockless and the difference
with your filesystem. We're talking about a lock being set to protect
the decoders context right.
So please give me some more info. Your filesystem is like:
static void yourfs_read(..... some parameters like inode number and request)
some translation from ino to path
do some more here maybe like creation of buffer
pthread_lock_mutex(some block var)
nreturn=call_to_your_backend(pathinformation, buffer)
pthread_unlock_mutex(some block var)
fuse_reply_buf(request, buffer......)
or fuse_reply_err when error
}
Post by Michael McTernanPost by Stef BonSo I suggest the problem lies with the decoder,
No. The problem stems from the current FUSE thread model.
Read/write requests can and do arrive out of order due to the
non-deterministic scheduling of threads. Unless a file system wishes to
implement a delay and a re-order buffer (e.g. using Nagle's algorithm),
there is currently nothing that can be done at the high level API to
prevent this behaviour and random access has to be implemented. Clearly
adding a delay for re-ordering each non-sequential access will harm
random-access performance unnecessarily, but not having a buffer may
lead to additional seeking which also has a cost.
I understand, please give some more info. In my filesystem
(fuse-workspace) I'm using a threads and a structure of mutex vars and
cond vars to have create some priority scheduling combined with random
access.
My filesystem offers a layer around the normal system:
/Computer
/Home
/Internet Services
/Mounts
/Network
/Shared
/System
The normal entries like bin, dev and etc and usr are present but
hidden ( and after doing a bind mount to the original fs and a chroot
you get this)
Now in the computer map there is hardware like removable devices like
USB sticks. It's removable, so at any time tge user can pull it out
(bluntly) or give a "safe remove" command. I'm currently working on
this issue to make this work.
When the user gives a safe remove command, this will result in a
"detach" of the USB device. I'm thinking of blocking any access to
the device during some period, like 15 seconds. When period is over,
device has become available again like normal, but during the 15
seconds the user can safely pull it out.
To achieve I added some extta fields to a structure. This structure
contains information about the resource, in this case a USB device,
but at can also be a SMB share, a cdrom or just a directory or file on
the localhost:
struct primary_resource_struct {
md5key_resource_string md5key;
int fd;
struct fd_list_struct *fd_list;
pathstring recordpath;
struct primary_resource_struct *next;
struct primary_resource_struct *prev;
struct mount_data_struct *mount_data;
unsigned char security;
unsigned char directio;
unsigned char userpolicy;
unsigned char status;
unsigned char actions;
pthread_mutex_t actions_mutex;
pthread_cond_t actions_condition;
unsigned char globallock;
pthread_mutex_t globallock_mutex;
pthread_cond_t globallock_condition;
unsigned group;
union {
struct local_cdrom_struct *cdrom;
struct local_disk_struct *disk;
struct net_smb_struct *smb;
struct local_map_struct *map;
struct local_file_struct *file;
} data;
};
There are a lot of fields, but important here are the actions var, and
the related actions_mutex and actions_condition, and the globallock
and the related globallock_mutex and globallock_condition.
A structure is created for the USB stick at:
/Computer/161-Transcend JetFlash Flash Drive
(the entry for this directory points indirectly to the primary_resource struct).
Now when a normal operation like read uses this device - like read -
it increases the actions paramater with one whe it starts the
operation, and decreases it again by one when ready.
So the parameter actions gives always the current number of normal operations.
It cannot increase just like that (see it like access), when the
status variable is not normal, something like "detached" or "removed"
it cannot increase, so access is denied.
The admin actions (like setting it in a stat of detached or back
again) (which instead of the normal operations which work on a single
entry) work on the whole device and use therefore the globallock var:
try to get a globallock, and set status to the desired value.
So when setting this to a value of DETACHED or REMOVED access is denied.
The remove thing is a process to finally remove the directory. In the
meantime - when this process is taking place - this status is very
functional.
With your decode you have to something simluar I guess.
But before I can say anything about that give some more info.
Stef