| # |
Name |
Description |
Links |
Comments from others |
DMF - End-User |
| 57. |
DMF clients for Windows |
There is no Windows Explorer interface to DMF, to allow the
inspection of file states and the requesting of recalls or migrations.
|
|
Vigorously discussed during a break at the Dec 2013 meeting.
From a site in Queensland: "It is probably our biggest
cause of user requests & complaints. (Windows users using
samba/cifs network drives, trying to access offline files.)"
Implemented at ISSP 3.4
|
| 58. |
DMF client programs should support CIFS-mounts
|
Some sites export filesystems via CIFS rather than NFS, because they need
to avoid UID mapping or they prefer authentication on a per-user basis.
The DMF clients for Linux and MacOSX do not support CIFS mounting.
|
2011 wish #28
|
|
| 59. |
Manage multiple files as a group |
Expressed in many different ways:
- migration/recall of directories
- control over the ordering of recalls to increase proximity
on tape
- "bucketing" small files into a collection, possibly using
tar, which is then recalled/migrated as a single item
|
2011 wish #22
|
|
| 60. |
Allow dmput of files by non-owners
|
Allow dmput of files by non-owners if they have read access;
this is necessary to prevent space quota problems for the owner.
|
2011 wish #25
|
|
DMF - Management & Operation |
| 61. |
Filesystem snapshot capability
|
Filesystems like ZFS and NetApp WAFL have a means of providing a
dated directory structure with snapshots providing a view of how
the filesystem would have looked at that time, so a user could
easily navigate to their own data and copy back a version of the
file they desire.
|
|
CSIRO's rsync backup system provides this and it's an amazingly
useful system management tool.
See here.
|
| 62. |
Excessive log entries for individual recall requests
|
Ability to easily parse logs for request data; currently it seems
that certain client side accesses produce multiple requests for the
same file (partial accesses, etc), which can then look like hundreds
or thousands of requests when in reality it is effectively just
one request/recall. This would allow for easier load monitoring
for requests.
|
|
Would also reduce the size of log files.
|
| 63. |
SGI patch numbers
|
SupportFolio and SGI staff refer to software updates by patch numbers, but
these are not visible through YAST or RPM. If you can't have a uniform and
consistent naming system for updates, then please make it easy to convert from
one format to another. Currently, you can install an update but not know
whether a particular patch is present on your machine or not.
|
|
|
| 64. |
Installation wizard |
Make an appliance-like wizard for new DMF installations
|
|
Technology Preview at ISSP 3.2
|
| 65. |
High-level (Python) programming API |
Currently our scripts parse output of the administrative commands, but
it would be nice if they could be programmed in a more direct way.
|
|
Perl too?
|
| 66. |
Control of daemon requests |
Provide visibility of and control over the DMF request queue to reprioritise
or cancel certain requests, recalls in particular.
Should include a CLI interface for scripting.
|
2011 wish #2
2012 wish #33
|
Implemented in stages from ISSP 3.0 - 3.4
|
| 67. |
Automatically prioritise DMF requests by type
|
In addition to manual control (previous item) also set default priorities
according to urgency and request type. For example:
- migrations when filesystem filling
- interactive recalls
- other recalls
- migrations when filesystem not filling
|
2011 wish #12
|
|
| 68. |
Easing generational media change
|
It would ease the movement of data from an old tape architecture
to a newer one if there was a standard tool which allowed you
to just specify the tapes to be migrated, either by VSN(s) or
by VG ownership. Ideally, it would be sensitive to the current
production load to avoid competing with it for resources.
|
Pres'n from Nov 2011
|
|
| 69. |
dmfdaemon doesn't know about dmatsnf |
It's odd that dmstat doesn't see this command. You can even disable all
tape movers and dmatsnf will still happily mount a volume and do its thing.
|
|
|
| 70. |
Removal of old drives from "dmstat -t"
|
To force "dmstat -t" to forget decommissioned tape drives, you have to
shut down DMF and then edit /dmf/spool/ls/dg/DriveStats.dump. This should
be documented, or better still, a cleaner method should be implemented.
|
|
|
| 71. |
Automatic tape import |
Automatic import of tapes into a volume group or allocation group based
on cartridge ID regular expression. Possibly a change to dmov_loadtapes
which could then run under cron.
This may be easy for a site to implement for themselves, but they
shouldn't have to.
|
|
|
| 72. |
Automatic tape export |
Automatic export of tapes based on policy. ie: "This tape is in the volume
group something and is full... export me at this time".
Possibly a wrapper around dmvoladm and ov_eject, run from cron.
This may be easy for a site to implement for themselves, but they
shouldn't have to.
|
|
|
| 73. |
Avoid & repair faulty tape chunks |
When a recall finds that a chunk cannot be read from tape, future recalls
should attempt to use other copies where possible. The faulty one should
be automatically regenerated later, as a background activity.
|
2011 wish #7
|
|
| 74. |
What file is currently being moved to/from tape?
|
It would be useful to have the ability to see what file (based the
original filename in the DMF managed directory) is currently either
being written to tape or retrieved from tape.
|
|
|
| 75. |
Listing the names of files on a tape
|
Given a tape VSN, it would be useful to have a list of files partly or
wholly on it. Looking for the name specifically (possibly the full path).
|
|
|
| 76. |
A single DMF database
|
Ability to collect information about dm*adm databases through a
single query/command; now one may need to query dmvoladm, then use
the results to query dmcatadm, etc.
This will allow more complicated queries to be built-in as standard.
For example, "what tapes have copies of the file with BFID xxx?"
|
Status at Dec 2013
|
Released as part of the PostgreSQL feature in DMF 6.9 (same timeframe as ISSP 3.4) for early adopters
|
| 77. |
Distributed DMF/OV databases
|
Some installations would like distributed DMF and OV databases such
that they could be shared by multiple DMF installations, either two
or more remote locations or two or more local locations.
You could then have an asset placed in a local library and another
copy placed in a remote one. You could also be able to retrieve that
asset from either location, without having to duplicate it.
|
|
Required for active-active HA?
|
| 78. |
Read-only access to the databases
|
The dm*adm commands should have a flag which says you do not want to
change the databases. Apart from providing safety, it would reduce
interlock issues between concurrent DB accessors. Potentially it would
allow read-only access to non-root users, though this should be
restricted to a username and/or group specified in dmf.conf
|
|
|
| 79. |
Support for end-to-end checksumming |
A customer of ours checksums all files and stores the checksum at their
site. Next, the file is copied over the net to our site and migrated
by DMF. At all stages we want to be able to verify the checksum,
preferably without re-reading the data.
It's proven difficult to solve this in an efficient way. One idea is
if DMF provided some hooks in the right places where site-specific
code would have access to the in-memory data buffers, it might be
possible to have real end-to-end checksumming.
|
|
In addition, DMF should also provide its own hashing to the user/admin to
reduce the same data being checksummed over and over by different parties
|
| 80. |
Tape integrity
|
Extend use of logical block protection (DIV) on T10000C drives and
future LTO when implemented with either an extension to dmatsnf or a
new command that uses a tape drive to verify the data integrity of an
entire tape and/or selected chunks. For a tape written with logical
block protection on, run a command to load tape and verify contents
using LBP/DIV options on the tape drive and report any errors.
This would allow the automation of verifying tapes that have not
been read/written for some time. The other benefit is that is only
requires access to a tape drive and does not impose any load on DMF
or the server, all the work is done on the tape drive.
|
|
|
Backups |
| 81. |
Non-DMF use of the VOL database |
Allow safe non-DMF use of the VOL database to manage tapes used
for backups. Ideally would include extra fields reserved for site use.
|
2012 wish #34
2012 wish #54
|
Site tables possible in PostgreSQL, released in DMF 6.9 (same timeframe as ISSP 3.4) for early adopters
|
| 82. |
DMF tapes should be self-sufficient
|
DMF tapes should be self-sufficient; now one needs both DMF DB
and inode data to be able to restore files from tapes; it would be
acceptable to only require a single, easily installable application
that one points to a given tape, and that could then be used to
restore files (all or selectively)
|
|
This would make it hard to handle file moves and renames. SAM-FS
has (or at least had) this problem.
|
| 83. |
xfsdump scalability for large numbers of inodes |
When you reach > 20M inodes in use in a filesystem, xfsdump takes a long
time (9 hours), even with a hybrid filesystem that puts the meta-data on SSD.
Filesystems with only 12M inodes in use get dumped in only 2 hours,
suggesting that the increase in duration is non-linear.
This is a big issue for us and we urge users to use "tar" in order to bring
down the number of files in a filesystem. See also wishlist item "bucketing".
|
2011 wish #17
|
Speedup in directory dumping implemented at ISSP 3.4
|
OpenVault/TMF/Tapes/MAID |
| 84. |
Centralized OV configuration
|
Ability to define (add/remove) LCP/DCPs just once and not for each
META/PDM node (for locally attached stuff ov_admin would talk to
each node and manage the sessions); similarly ability to see the
definitions from each node in one place ("do all nodes have the
correct/identical configuration")
|
Status at Dec 2012
|
|
| 85. |
Show tape position in OpenVault. |
Real-time tape position data from OpenVault, like TMF's tmstat
has, would be a valuable debugging aid.
This may involve adding instrumentation to the ts (and one day,
st) kernel modules.
|
2011 wish #30
|
|
Documentation |
| 86. |
Size of dmf.conf man page
|
The dmf.conf man page has grown so much it's becoming hard to use.
Perhaps some content could be moved to man pages of their own? The
descriptions of the maintenance tasks would be a good starting point.
|
|
|
| 87. |
FC switch zoning |
Should "Best Practices" in the Admin Guide include recommendations for FC
switch zoning? Currently, it's only briefly mentioned in the Introduction.
|
|
|
| 88. |
Creating /etc/failover2.conf |
Should "Best Practices" in the Admin Guide describe the use of mk_failover2.pl?
It doesn't seem to be documented, and can save a lot of effort.
|
|
|
| 89. |
Links in PDF manuals
|
Add internal links/references to the pages within the DMF related manuals.
A section or page reference in the text should allow you click on that
reference to go to that page/section.
|
|
|
