Files would be skipped from the backup. Convert unmappable characters File not found errors in the backup logs.

(Last modified: 19Sep2005)

This document (10097645) is provided subject to the disclaimer at the end of this document.

fact

NetWare 6.5 sp2

NetWare 6.5 sp3

NetWare 6.5 sp4

Tape backup software

Unicode UTF8 support

symptom

Files would be skipped from the backup. Convert unmappable characters File not found errors in the backup logs.

tsafs /(no)ConvertUnmappableChars

cause

In NSS, the file names are stored in Unicode format. SMS (Storage Management Services) uses the unicode names during backup and restore for the NSS file system to avoid conversion of names and hence the loss in certain character conversions.

 
 This support for unicode names in SMS is available in TSAFS and is supported in NW6.x and upwards. For SMS to use the unicode names, the backup application interface should inform SMS to do so by passing the required flags. There are backup applications which do not pass these flags causing the SMS to perform an MBCS to Unicode conversion  thereby causing loss in character information.
 
 During this conversion process, certain characters are not mapped properly causing the NSS to interpret these files as unavailable and hence returning errors to the backup application about the non-availability of such files and directories.
 
 For this to work, the backup application should change to use the UTF8 interface of SMS and provide necessary data for SMS to process the file names properly. More details can be found in the documentation provided at the following URL
http://developer.novell.com/ndk/doc/smscomp/index.html?page=/ndk/doc/smscomp/sms_reco.pdf
 
 When the backup application uses the non-UTF8 interface (original interface) of SMS, then LibC will convert the un-mappable characters into equivalent unicode representations, represented within square brackets with a 4 digit sequence. This 4 digit special sequence is then converted back to the original character by the LibC, as SMS uses the LibC for character conversions. However, some file names might contain this sequence as part of the name, for example certain files have the year represented as this sequence [2001], which might match the LibC generated unicode sequence. Since this sequence is present in the file name and is part of the file name, LibC interprets it as a special sequence and converts it into a character, resulting in a filename that does not exist in the file system. To avoid this issue, TSAFS has provided a switch called /noConvertUnMappableChars.
 
 This switch can be used as a workaround to avoid the mis-interpreting of characters that are not special sequence instead part of the file name itself.
 

fix

Startup Switch Name:
tsafs /NoConvertUnMappableChars
 
How to use
The name of the switch is ConvertUnMappableChars. By default this switch will be set to TRUE, so that normal backups and restores will not be affected. When this switch is set to FALSE, TSAFS will re-try to access the same file by ignoring the mapping algorithm for converting the unmappable characters. 

 load tsafs /noConvertUnMappableChars  to disable the default mapping. This is a load time parameter.

 Logging in tsafs can be used to determine if this problem is happening. Backing up just a few files that have the problem would be sufficient. Otherwise the tsadbg.log file can grow to a very large size resulting in low disk space.

To enable tsafs logging:

tsafs /smsdebug=fffffffc /smsdebug2=fffffffc

To turn off the logging:

tsafs /smsdebug=0 /smsdebug2=0

This will create the sys:system\tsa\tsadbg.log file.


The real solution however, needs to come from the tape software vendor.  Utf8 unicode support needs to be added to the tape software. This load line parameter is a potential workaround.

document

Document Title: Files would be skipped from the backup. Convert unmappable characters File not found errors in the backup logs.
Document ID: 10097645
Solution ID: NOVL102059
Creation Date: 09May2005
Modified Date: 19Sep2005
Novell Product Class:Netware

disclaimer

The Origin of this information may be internal or external to Novell. Novell makes all reasonable efforts to verify this information. However, the information provided in this document is for your information only. Novell makes no explicit or implied claims to the validity of this information.
Any trademarks referenced in this document are the property of their respective owners. Consult your product manuals for complete trademark information.