Discussion:
Read performance issue when upgrading from FUSE 2.9.4 to 3.0.2 (readahead not working?)
(too old to reply)
David Sorber
2017-06-07 15:09:05 UTC
Permalink
I've been working on a FUSE-based file system using FUSE 2.9.4 on Ubuntu
14.04 for a while now. It works well and is reasonably stable. Recently I
decided it might be a good idea to upgrade to FUSE 3. I made the necessary
interface changes, installed FUSE 3, and everything works nicely.

However, I noticed a performance regression after upgrading to FUSE 3. I
wrote a very simple test utility that calls fgets() in a loop and iterates
over a text file that is ~120 MB. Using the previous revision that uses
FUSE 2, the test utility runs in ~200 ms. When I switch to the FUSE 3
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3 instead of
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE 3.0.2
installed side-by-side on the same machine for testing.

I added some additional debug logging and upon closer inspection I noticed
that for the FUSE 2 revision I see ~4-5 read requests per millisecond. For
the FUSE 3 revision I see only 1 read request every ~20 milliseconds. The
time to complete each read appears to be the same across both revisions.
This leads me to wonder (and maybe I'm totally wrong about this...) if
somehow the kernel block layer's readahead isn't working correctly, or
isn't configured when I'm using FUSE 3? Does anyone have any ideas as to
what may be wrong? or other things to look at?


Below is the initial startup debug output for both revisions:

FUSE library version: 2.9.4
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
FUSE library version: 2.9.4
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.23
flags=0x0003f7fb
max_readahead=0x00020000
INIT: 7.19
flags=0x00000031
max_readahead=0x00020000
max_write=0x00020000
max_background=0
congestion_threshold=0
unique: 1, success, outsize: 40
unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 7065
getattr /
unique: 2, success, outsize: 120




FUSE library version: 3.0.2
nullpath_ok: 0
FUSE library version: 3.0.2
nullpath_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.23
flags=0x0003f7fb
max_readahead=0x00020000
INIT: 7.26
flags=0x0000d031
max_readahead=0x00020000
max_write=0x00020000
max_background=0
congestion_threshold=0
time_gran=0
unique: 1, success, outsize: 80
unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 8531
getattr[NULL] /
unique: 2, success, outsize: 120


Things look to be configured identically as far as I can tell.


Any thoughts or suggestions would be appreciated.

Thanks,
-David
Nikolaus Rath
2017-06-07 17:02:31 UTC
Permalink
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on Ubuntu
14.04 for a while now. It works well and is reasonably stable. Recently I
decided it might be a good idea to upgrade to FUSE 3. I made the necessary
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to FUSE 3. I
wrote a very simple test utility that calls fgets() in a loop and iterates
over a text file that is ~120 MB. Using the previous revision that uses
FUSE 2, the test utility runs in ~200 ms. When I switch to the FUSE 3
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3 instead of
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE 3.0.2
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
fuse_common.h):

/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the attributes
* on every read. If the attributes are no longer valid (i.e., if the
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file changes
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)


Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«
David Sorber
2017-06-07 17:38:50 UTC
Permalink
Nikolaus,

Thanks for pointing me in the right direction. FUSE_CAP_AUTO_INVAL_DATA
appears to be the culprit. (Although my FUSE 3 revision is now running in
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I can
live with that for now).

I am setting attr_timeout to zero (all my file changes do indeed go through
the kernel), but I guess the attribute validity check on every read was
causing the slowdown. I missed that detail.

Thanks again.

-David
Post by David Sorber
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on Ubuntu
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to FUSE 3. I
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision that uses
FUSE 2, the test utility runs in ~200 ms. When I switch to the FUSE 3
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3 instead
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE 3.0.2
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the attributes
* on every read. If the attributes are no longer valid (i.e., if the
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file changes
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
Nikolaus Rath
2017-06-07 22:03:42 UTC
Permalink
Hi David,

If you set attr_timeout to zero, you should *not* see getattr() calls on
every read. Can you confirm?

Best,
Nikolaus

On Jun 07 2017, David Sorber <david.sorber-***@public.gmane.org> wrote:
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction. FUSE_CAP_AUTO_INVAL_DATA
appears to be the culprit. (Although my FUSE 3 revision is now running in
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I can
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed go through
the kernel), but I guess the attribute validity check on every read was
causing the slowdown. I missed that detail.
Thanks again.
-David
Post by David Sorber
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on Ubuntu
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to FUSE 3. I
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision that uses
FUSE 2, the test utility runs in ~200 ms. When I switch to the FUSE 3
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3 instead
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE 3.0.2
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the attributes
* on every read. If the attributes are no longer valid (i.e., if the
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file changes
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«
David Sorber
2017-06-08 11:55:31 UTC
Permalink
Nikolaus,

I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...

With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set to zero
inside of init(), I see a large number of calls to getattr() (~250) in
between each read request. It appears that something may be broken.

Please let me know if/how I can assist.

Thanks,
-David
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr() calls on
every read. Can you confirm?
Best,
Nikolaus
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction. FUSE_CAP_AUTO_INVAL_DATA
appears to be the culprit. (Although my FUSE 3 revision is now running
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I can
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed go
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every read was
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to FUSE
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision that
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to the FUSE 3
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the attributes
* on every read. If the attributes are no longer valid (i.e., if the
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file changes
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
Nikolaus Rath
2017-06-08 16:45:51 UTC
Permalink
Hi David,

Apologies, I was blindly trusting the documentation. On closer
inspection, this doesn't make any sense. A timeout of zero will of
course force a getattr() on every read. However, if all your file
systems changes go through the kernel, then you should not be using a
timeout of zero, but a something very large. This should then also fix
the getattr() issue without needing to unset FUSE_CAP_AUTO_INVAL_DATA.

I will fix the documentation ASAP.

Best,
Nikolaus
Post by David Sorber
Nikolaus,
I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...
With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set to zero
inside of init(), I see a large number of calls to getattr() (~250) in
between each read request. It appears that something may be broken.
Please let me know if/how I can assist.
Thanks,
-David
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr() calls on
every read. Can you confirm?
Best,
Nikolaus
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction. FUSE_CAP_AUTO_INVAL_DATA
appears to be the culprit. (Although my FUSE 3 revision is now running
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I can
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed go
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every read was
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to FUSE
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision that
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to the FUSE 3
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the attributes
* on every read. If the attributes are no longer valid (i.e., if the
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file changes
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«
David Sorber
2017-06-08 17:35:20 UTC
Permalink
Nikolaus,

I tried what you describe above, set FUSE_CAP_AUTO_INVAL_DATA and make
attr_timeout
very large, earlier today and I still see one getattr() call between each
read no matter how large I make attr_timeout.

I am okay with disabling FUSE_CAP_AUTO_INVAL_DATA, is there a reason I
should want to enable it if all of my file system changes are going through
the kernel?

Thanks,
-David
Post by Nikolaus Rath
Hi David,
Apologies, I was blindly trusting the documentation. On closer
inspection, this doesn't make any sense. A timeout of zero will of
course force a getattr() on every read. However, if all your file
systems changes go through the kernel, then you should not be using a
timeout of zero, but a something very large. This should then also fix
the getattr() issue without needing to unset FUSE_CAP_AUTO_INVAL_DATA.
I will fix the documentation ASAP.
Best,
Nikolaus
Post by David Sorber
Nikolaus,
I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...
With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set to
zero
Post by David Sorber
inside of init(), I see a large number of calls to getattr() (~250) in
between each read request. It appears that something may be broken.
Please let me know if/how I can assist.
Thanks,
-David
On Wed, Jun 7, 2017 at 6:03 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr() calls on
every read. Can you confirm?
Best,
Nikolaus
On Jun 07 2017, David Sorber <david.sorber-
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction.
FUSE_CAP_AUTO_INVAL_DATA
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
appears to be the culprit. (Although my FUSE 3 revision is now
running
Post by David Sorber
Post by Nikolaus Rath
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I
can
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed go
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every read
was
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to FUSE
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision that
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to the
FUSE 3
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
revision the same utility now takes ~30,000 ms. Note that the only
modifications I made were interface modifications to use FUSE 3
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and FUSE
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the
attributes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* on every read. If the attributes are no longer valid (i.e., if the
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file changes
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
Michael Theall
2017-06-08 17:46:33 UTC
Permalink
Hi Nikolaus/David,

It sounds to me like you *don't* want to use FUSE_CAP_AUTO_INVAL_DATA. As
far as I can tell, without setting this and by using a non-zero attr
timeout you should only get a GETATTR when you reach the end of the file or
the attr has timed out.

Regards,
Michael Theall
Post by David Sorber
Nikolaus,
I tried what you describe above, set FUSE_CAP_AUTO_INVAL_DATA and make attr_timeout
very large, earlier today and I still see one getattr() call between each
read no matter how large I make attr_timeout.
I am okay with disabling FUSE_CAP_AUTO_INVAL_DATA, is there a reason I
should want to enable it if all of my file system changes are going through
the kernel?
Thanks,
-David
Post by Nikolaus Rath
Hi David,
Apologies, I was blindly trusting the documentation. On closer
inspection, this doesn't make any sense. A timeout of zero will of
course force a getattr() on every read. However, if all your file
systems changes go through the kernel, then you should not be using a
timeout of zero, but a something very large. This should then also fix
the getattr() issue without needing to unset FUSE_CAP_AUTO_INVAL_DATA.
I will fix the documentation ASAP.
Best,
Nikolaus
On Jun 08 2017, David Sorber <
Post by David Sorber
Nikolaus,
I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...
With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set to
zero
Post by David Sorber
inside of init(), I see a large number of calls to getattr() (~250) in
between each read request. It appears that something may be broken.
Please let me know if/how I can assist.
Thanks,
-David
On Wed, Jun 7, 2017 at 6:03 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr() calls
on
Post by David Sorber
Post by Nikolaus Rath
every read. Can you confirm?
Best,
Nikolaus
On Jun 07 2017, David Sorber
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction.
FUSE_CAP_AUTO_INVAL_DATA
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
appears to be the culprit. (Although my FUSE 3 revision is now
running
Post by David Sorber
Post by Nikolaus Rath
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I
can
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed go
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every read
was
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to
FUSE
Post by David Sorber
Post by Nikolaus Rath
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision
that
Post by David Sorber
Post by Nikolaus Rath
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to the
FUSE 3
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
revision the same utility now takes ~30,000 ms. Note that the
only
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
modifications I made were interface modifications to use FUSE 3
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and
FUSE
Post by David Sorber
Post by Nikolaus Rath
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the
attributes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* on every read. If the attributes are no longer valid (i.e., if
the
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file
changes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Nikolaus Rath
2017-06-08 20:58:19 UTC
Permalink
Hi Michael,

Disabling FUSE_CAP_AUTO_INVAL_DATA is a bad idea, because it globally
bypasses the timeout.

Setting a non-zeri attr timeout should be sufficient to avoid pointless
getattrs().


Best,
Nikolaus
Post by Michael Theall
Hi Nikolaus/David,
It sounds to me like you *don't* want to use FUSE_CAP_AUTO_INVAL_DATA. As
far as I can tell, without setting this and by using a non-zero attr
timeout you should only get a GETATTR when you reach the end of the file or
the attr has timed out.
Regards,
Michael Theall
On Thu, Jun 8, 2017 at 12:35 PM David Sorber
Post by David Sorber
Nikolaus,
I tried what you describe above, set FUSE_CAP_AUTO_INVAL_DATA and make attr_timeout
very large, earlier today and I still see one getattr() call between each
read no matter how large I make attr_timeout.
I am okay with disabling FUSE_CAP_AUTO_INVAL_DATA, is there a reason I
should want to enable it if all of my file system changes are going through
the kernel?
Thanks,
-David
Post by Nikolaus Rath
Hi David,
Apologies, I was blindly trusting the documentation. On closer
inspection, this doesn't make any sense. A timeout of zero will of
course force a getattr() on every read. However, if all your file
systems changes go through the kernel, then you should not be using a
timeout of zero, but a something very large. This should then also fix
the getattr() issue without needing to unset FUSE_CAP_AUTO_INVAL_DATA.
I will fix the documentation ASAP.
Best,
Nikolaus
On Jun 08 2017, David Sorber <
Post by David Sorber
Nikolaus,
I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...
With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set to
zero
Post by David Sorber
inside of init(), I see a large number of calls to getattr() (~250) in
between each read request. It appears that something may be broken.
Please let me know if/how I can assist.
Thanks,
-David
On Wed, Jun 7, 2017 at 6:03 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr() calls
on
Post by David Sorber
Post by Nikolaus Rath
every read. Can you confirm?
Best,
Nikolaus
On Jun 07 2017, David Sorber
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction.
FUSE_CAP_AUTO_INVAL_DATA
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
appears to be the culprit. (Although my FUSE 3 revision is now
running
Post by David Sorber
Post by Nikolaus Rath
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess I
can
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed go
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every read
was
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4 on
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made the
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works nicely.
However, I noticed a performance regression after upgrading to
FUSE
Post by David Sorber
Post by Nikolaus Rath
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop and
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision
that
Post by David Sorber
Post by Nikolaus Rath
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to the
FUSE 3
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
revision the same utility now takes ~30,000 ms. Note that the
only
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
modifications I made were interface modifications to use FUSE 3
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and
FUSE
Post by David Sorber
Post by Nikolaus Rath
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that the
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module only
* asks the filesystem for an update of the file's attributes when a
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the
attributes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* on every read. If the attributes are no longer valid (i.e., if
the
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file
changes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the kernel.
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«
Michael Theall
2017-06-08 21:46:49 UTC
Permalink
Hi Nikolaus,

Are you sure about that? I only see auto_inval_data used in two places in
the kernel:

1) fuse_change_attributes: invalidate attributes if changing mtime
2) fuse_file_read_iter: refresh attributes before every read (if disabled
only care if we are reading past end-of-file)

Regards,
Michael Theall
Post by Nikolaus Rath
Hi Michael,
Disabling FUSE_CAP_AUTO_INVAL_DATA is a bad idea, because it globally
bypasses the timeout.
Setting a non-zeri attr timeout should be sufficient to avoid pointless
getattrs().
Best,
Nikolaus
On Jun 08 2017, Michael Theall <
Post by Michael Theall
Hi Nikolaus/David,
It sounds to me like you *don't* want to use FUSE_CAP_AUTO_INVAL_DATA. As
far as I can tell, without setting this and by using a non-zero attr
timeout you should only get a GETATTR when you reach the end of the file
or
Post by Michael Theall
the attr has timed out.
Regards,
Michael Theall
On Thu, Jun 8, 2017 at 12:35 PM David Sorber
Post by David Sorber
Nikolaus,
I tried what you describe above, set FUSE_CAP_AUTO_INVAL_DATA and make
attr_timeout
Post by Michael Theall
Post by David Sorber
very large, earlier today and I still see one getattr() call between
each
Post by Michael Theall
Post by David Sorber
read no matter how large I make attr_timeout.
I am okay with disabling FUSE_CAP_AUTO_INVAL_DATA, is there a reason I
should want to enable it if all of my file system changes are going
through
Post by Michael Theall
Post by David Sorber
the kernel?
Thanks,
-David
On Thu, Jun 8, 2017 at 12:45 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
Apologies, I was blindly trusting the documentation. On closer
inspection, this doesn't make any sense. A timeout of zero will of
course force a getattr() on every read. However, if all your file
systems changes go through the kernel, then you should not be using a
timeout of zero, but a something very large. This should then also fix
the getattr() issue without needing to unset FUSE_CAP_AUTO_INVAL_DATA.
I will fix the documentation ASAP.
Best,
Nikolaus
On Jun 08 2017, David Sorber <
Post by David Sorber
Nikolaus,
I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...
With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set to
zero
Post by David Sorber
inside of init(), I see a large number of calls to getattr() (~250)
in
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
between each read request. It appears that something may be broken.
Please let me know if/how I can assist.
Thanks,
-David
On Wed, Jun 7, 2017 at 6:03 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr()
calls
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
on
Post by David Sorber
Post by Nikolaus Rath
every read. Can you confirm?
Best,
Nikolaus
On Jun 07 2017, David Sorber
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction.
FUSE_CAP_AUTO_INVAL_DATA
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
appears to be the culprit. (Although my FUSE 3 revision is now
running
Post by David Sorber
Post by Nikolaus Rath
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I guess
I
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
can
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed
go
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every
read
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
was
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE 2.9.4
on
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably stable.
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made
the
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works
nicely.
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
However, I noticed a performance regression after upgrading to
FUSE
Post by David Sorber
Post by Nikolaus Rath
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop
and
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision
that
Post by David Sorber
Post by Nikolaus Rath
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to the
FUSE 3
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
revision the same utility now takes ~30,000 ms. Note that the
only
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
modifications I made were interface modifications to use FUSE 3
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and
FUSE
Post by David Sorber
Post by Nikolaus Rath
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that
the
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module
only
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* asks the filesystem for an update of the file's attributes
when a
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the
attributes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* on every read. If the attributes are no longer valid (i.e., if
the
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a `getattr`
* request. If the new mtime differs from the previous value, any
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file
changes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* go through the kernel, *attr_timeout* should be set to zero to
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the
kernel.
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E
599F
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
»Time flies like an arrow, fruit flies like a
Banana.«
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------------------------
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by Michael Theall
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by Michael Theall
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Michael Theall
2017-06-08 21:58:09 UTC
Permalink
Hi Nikolaus,

Sorry I think I see what you're saying now. fuse_update_attributes will
check for that timeout. I don't see why David would be getting GETATTR
requests unless he's explicitly invalidating attributes or otherwise the
timeout has passed.

Regards,
Michael Theall
Post by Michael Theall
Hi Nikolaus,
Are you sure about that? I only see auto_inval_data used in two places in
1) fuse_change_attributes: invalidate attributes if changing mtime
2) fuse_file_read_iter: refresh attributes before every read (if disabled
only care if we are reading past end-of-file)
Regards,
Michael Theall
Post by Nikolaus Rath
Hi Michael,
Disabling FUSE_CAP_AUTO_INVAL_DATA is a bad idea, because it globally
bypasses the timeout.
Setting a non-zeri attr timeout should be sufficient to avoid pointless
getattrs().
Best,
Nikolaus
On Jun 08 2017, Michael Theall <
Post by Michael Theall
Hi Nikolaus/David,
It sounds to me like you *don't* want to use FUSE_CAP_AUTO_INVAL_DATA.
As
Post by Michael Theall
far as I can tell, without setting this and by using a non-zero attr
timeout you should only get a GETATTR when you reach the end of the
file or
Post by Michael Theall
the attr has timed out.
Regards,
Michael Theall
On Thu, Jun 8, 2017 at 12:35 PM David Sorber
Post by David Sorber
Nikolaus,
I tried what you describe above, set FUSE_CAP_AUTO_INVAL_DATA and make
attr_timeout
Post by Michael Theall
Post by David Sorber
very large, earlier today and I still see one getattr() call between
each
Post by Michael Theall
Post by David Sorber
read no matter how large I make attr_timeout.
I am okay with disabling FUSE_CAP_AUTO_INVAL_DATA, is there a reason I
should want to enable it if all of my file system changes are going
through
Post by Michael Theall
Post by David Sorber
the kernel?
Thanks,
-David
On Thu, Jun 8, 2017 at 12:45 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
Apologies, I was blindly trusting the documentation. On closer
inspection, this doesn't make any sense. A timeout of zero will of
course force a getattr() on every read. However, if all your file
systems changes go through the kernel, then you should not be using a
timeout of zero, but a something very large. This should then also fix
the getattr() issue without needing to unset FUSE_CAP_AUTO_INVAL_DATA.
I will fix the documentation ASAP.
Best,
Nikolaus
On Jun 08 2017, David Sorber <
Post by David Sorber
Nikolaus,
I was a bit confused by this given the description in the
FUSE_CAP_AUTO_INVAL_DATA comment...
With FUSE_CAP_AUTO_INVAL_DATA enabled and config->attr_timeout set
to
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
zero
Post by David Sorber
inside of init(), I see a large number of calls to getattr() (~250)
in
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
between each read request. It appears that something may be broken.
Please let me know if/how I can assist.
Thanks,
-David
On Wed, Jun 7, 2017 at 6:03 PM, Nikolaus Rath <
Post by Nikolaus Rath
Hi David,
If you set attr_timeout to zero, you should *not* see getattr()
calls
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
on
Post by David Sorber
Post by Nikolaus Rath
every read. Can you confirm?
Best,
Nikolaus
On Jun 07 2017, David Sorber
g> Nikolaus,
Post by David Sorber
Thanks for pointing me in the right direction.
FUSE_CAP_AUTO_INVAL_DATA
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
appears to be the culprit. (Although my FUSE 3 revision is now
running
Post by David Sorber
Post by Nikolaus Rath
in
Post by David Sorber
~300 ms instead of the ~200 ms for the FUSE 2 revision... I
guess I
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
can
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
live with that for now).
I am setting attr_timeout to zero (all my file changes do indeed
go
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
through
Post by David Sorber
the kernel), but I guess the attribute validity check on every
read
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
was
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
causing the slowdown. I missed that detail.
Thanks again.
-David
On Wed, Jun 7, 2017 at 1:02 PM, Nikolaus Rath <
Post by Nikolaus Rath
On Jun 07 2017, David Sorber <david.sorber-
Post by David Sorber
I've been working on a FUSE-based file system using FUSE
2.9.4 on
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Ubuntu
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
14.04 for a while now. It works well and is reasonably
stable.
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Recently I
Post by David Sorber
decided it might be a good idea to upgrade to FUSE 3. I made
the
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
necessary
Post by David Sorber
interface changes, installed FUSE 3, and everything works
nicely.
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
However, I noticed a performance regression after upgrading to
FUSE
Post by David Sorber
Post by Nikolaus Rath
3. I
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
wrote a very simple test utility that calls fgets() in a loop
and
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
iterates
Post by David Sorber
over a text file that is ~120 MB. Using the previous revision
that
Post by David Sorber
Post by Nikolaus Rath
uses
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
FUSE 2, the test utility runs in ~200 ms. When I switch to
the
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
FUSE 3
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
revision the same utility now takes ~30,000 ms. Note that the
only
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
modifications I made were interface modifications to use FUSE
3
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
instead
Post by David Sorber
Post by Nikolaus Rath
of
Post by David Sorber
FUSE 2, no other code was changed. I have both FUSE 2.9.4 and
FUSE
Post by David Sorber
Post by Nikolaus Rath
3.0.2
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
installed side-by-side on the same machine for testing.
Are you setting up the capabilities in the same way? Note that
the
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
defaults have changed. In particular, I'd look at (from
/**
* Traditionally, while a file is open the FUSE kernel module
only
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* asks the filesystem for an update of the file's attributes
when a
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* client attempts to read beyond EOF. This is unsuitable for
* e.g. network filesystems, where the file contents may change
* without the kernel knowing about it.
*
* If this flag is set, FUSE will check the validity of the
attributes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* on every read. If the attributes are no longer valid (i.e.,
if
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
the
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* *attr_timeout* passed to fuse_reply_attr() or set in `struct
* fuse_entry_param` has passed), it will first issue a
`getattr`
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* request. If the new mtime differs from the previous value,
any
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* cached file *contents* will be invalidated as well.
*
* This flag should always be set when available. If all file
changes
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* go through the kernel, *attr_timeout* should be set to zero
to
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
* avoid unneccessary getattr() calls.
*
* This feature is enabled by default when supported by the
kernel.
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
*/
#define FUSE_CAP_AUTO_INVAL_DATA (1 << 12)
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E
599F
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
»Time flies like an arrow, fruit flies like a
Banana.«
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
lists/listinfo/fuse-devel
------------------------------------------------------------
------------------
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Post by Nikolaus Rath
lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/
lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------------------------
Post by Michael Theall
Post by David Sorber
Post by Nikolaus Rath
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by Michael Theall
Post by David Sorber
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
------------------------------------------------------------------------------
Post by Michael Theall
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Loading...