Reference for all built-in template language functions

Here, we document all the built-in functions available in the calibre template language. Every function is implemented as a class in python and you can click the source links to see the source code, in case the documentation is insufficient. The functions are arranged in logical groups by type.

Arithmetic

add(x, y)

class calibre.utils.formatter_functions.BuiltinAdd

add(x, y) – returns x + y. Throws an exception if either x or y are not numbers.

divide(x, y)

class calibre.utils.formatter_functions.BuiltinDivide

divide(x, y) – returns x / y. Throws an exception if either x or y are not numbers.

multiply(x, y)

class calibre.utils.formatter_functions.BuiltinMultiply

multiply(x, y) – returns x * y. Throws an exception if either x or y are not numbers.

subtract(x, y)

class calibre.utils.formatter_functions.BuiltinSubtract

subtract(x, y) – returns x - y. Throws an exception if either x or y are not numbers.

Boolean

and(value, value, ...)

class calibre.utils.formatter_functions.BuiltinAnd

and(value, value, ...) – returns the string “1” if all values are not empty, otherwise returns the empty string. This function works well with test or first_non_empty. You can have as many values as you want.

not(value)

class calibre.utils.formatter_functions.BuiltinNot

not(value) – returns the string “1” if the value is empty, otherwise returns the empty string. This function works well with test or first_non_empty. You can have as many values as you want.

or(value, value, ...)

class calibre.utils.formatter_functions.BuiltinOr

or(value, value, ...) – returns the string “1” if any value is not empty, otherwise returns the empty string. This function works well with test or first_non_empty. You can have as many values as you want.

Date functions

days_between(date1, date2)

class calibre.utils.formatter_functions.BuiltinDaysBetween

days_between(date1, date2) – return the number of days between date1 and date2. The number is positive if date1 is greater than date2, otherwise negative. If either date1 or date2 are not dates, the function returns the empty string.

today()

class calibre.utils.formatter_functions.BuiltinToday

today() – return a date string for today. This value is designed for use in format_date or days_between, but can be manipulated like any other string. The date is in ISO format.

Formatting values

finish_formatting(val, fmt, prefix, suffix)

class calibre.utils.formatter_functions.BuiltinFinishFormatting

finish_formatting(val, fmt, prefix, suffix) – apply the format, prefix, and suffix to a value in the same way as done in a template like {series_index:05.2f| - |- }. For example, the following program produces the same output as the above template: program: finish_formatting(field(“series_index”), “05.2f”, ” - ”, ” - ”)

format_date(val, format_string)

class calibre.utils.formatter_functions.BuiltinFormatDate

format_date(val, format_string) – format the value, which must be a date, using the format_string, returning a string. The formatting codes are: d : the day as number without a leading zero (1 to 31) dd : the day as number with a leading zero (01 to 31) ddd : the abbreviated localized day name (e.g. “Mon” to “Sun”). dddd : the long localized day name (e.g. “Monday” to “Sunday”). M : the month as number without a leading zero (1 to 12). MM : the month as number with a leading zero (01 to 12) MMM : the abbreviated localized month name (e.g. “Jan” to “Dec”). MMMM : the long localized month name (e.g. “January” to “December”). yy : the year as two digit number (00 to 99). yyyy : the year as four digit number. h : the hours without a leading 0 (0 to 11 or 0 to 23, depending on am/pm) hh : the hours with a leading 0 (00 to 11 or 00 to 23, depending on am/pm) m : the minutes without a leading 0 (0 to 59) mm : the minutes with a leading 0 (00 to 59) s : the seconds without a leading 0 (0 to 59) ss : the seconds with a leading 0 (00 to 59) ap : use a 12-hour clock instead of a 24-hour clock, with “ap” replaced by the localized string for am or pm AP : use a 12-hour clock instead of a 24-hour clock, with “AP” replaced by the localized string for AM or PM iso : the date with time and timezone. Must be the only format present

format_number(v, template)

class calibre.utils.formatter_functions.BuiltinFormatNumber

format_number(v, template) – format the number v using a python formatting template such as “{0:5.2f}” or “{0:,d}” or “${0:5,.2f}”. The field_name part of the template must be a 0 (zero) (the “{0:” in the above examples). See the template language and python documentation for more examples. Returns the empty string if formatting fails.

human_readable(v)

class calibre.utils.formatter_functions.BuiltinHumanReadable

human_readable(v) – return a string representing the number v in KB, MB, GB, etc.

Get values from metadata

approximate_formats()

class calibre.utils.formatter_functions.BuiltinApproximateFormats

approximate_formats() – return a comma-separated list of formats that at one point were associated with the book. There is no guarantee that this list is correct, although it probably is. This function can be called in template program mode using the template “{:’approximate_formats()’}”. Note that format names are always uppercase, as in EPUB.

booksize()

class calibre.utils.formatter_functions.BuiltinBooksize

booksize() – return value of the size field

current_library_name()

class calibre.utils.formatter_functions.BuiltinCurrentLibraryName

current_library_name() – return the last name on the path to the current calibre library. This function can be called in template program mode using the template “{:’current_library_name()’}”.

current_library_path()

class calibre.utils.formatter_functions.BuiltinCurrentLibraryPath

current_library_path() – return the path to the current calibre library. This function can be called in template program mode using the template “{:’current_library_path()’}”.

field(name)

class calibre.utils.formatter_functions.BuiltinField

field(name) – returns the metadata field named by name

formats_modtimes(date_format)

class calibre.utils.formatter_functions.BuiltinFormatsModtimes

formats_modtimes(date_format) – return a comma-separated list of colon_separated items representing modification times for the formats of a book. The date_format parameter specifies how the date is to be formatted. See the date_format function for details. You can use the select function to get the mod time for a specific format. Note that format names are always uppercase, as in EPUB.

formats_paths()

class calibre.utils.formatter_functions.BuiltinFormatsPaths

formats_paths() – return a comma-separated list of colon_separated items representing full path to the formats of a book. You can use the select function to get the path for a specific format. Note that format names are always uppercase, as in EPUB.

formats_sizes()

class calibre.utils.formatter_functions.BuiltinFormatsSizes

formats_sizes() – return a comma-separated list of colon_separated items representing sizes in bytes of the formats of a book. You can use the select function to get the size for a specific format. Note that format names are always uppercase, as in EPUB.

has_cover()

class calibre.utils.formatter_functions.BuiltinHasCover

has_cover() – return Yes if the book has a cover, otherwise return the empty string

language_codes(lang_strings)

class calibre.utils.formatter_functions.BuiltinLanguageCodes

language_codes(lang_strings) – return the language codes for the strings passed in lang_strings. The strings must be in the language of the current locale. Lang_strings is a comma-separated list.

language_strings(lang_codes, localize)

class calibre.utils.formatter_functions.BuiltinLanguageStrings

language_strings(lang_codes, localize) – return the strings for the language codes passed in lang_codes. If localize is zero, return the strings in English. If localize is not zero, return the strings in the language of the current locale. Lang_codes is a comma-separated list.

ondevice()

class calibre.utils.formatter_functions.BuiltinOndevice

ondevice() – return Yes if ondevice is set, otherwise return the empty string

raw_field(name)

class calibre.utils.formatter_functions.BuiltinRawField

raw_field(name) – returns the metadata field named by name without applying any formatting.

series_sort()

class calibre.utils.formatter_functions.BuiltinSeriesSort

series_sort() – return the series sort value

If-then-else

contains(val, pattern, text if match, text if not match)

class calibre.utils.formatter_functions.BuiltinContains

contains(val, pattern, text if match, text if not match) – checks if field contains matches for the regular expression pattern. Returns text if match if matches are found, otherwise it returns text if no match

ifempty(val, text if empty)

class calibre.utils.formatter_functions.BuiltinIfempty

ifempty(val, text if empty) – return val if val is not empty, otherwise return text if empty

test(val, text if not empty, text if empty)

class calibre.utils.formatter_functions.BuiltinTest

test(val, text if not empty, text if empty) – return text if not empty if the field is not empty, otherwise return text if empty

Iterating over values

first_non_empty(value, value, ...)

class calibre.utils.formatter_functions.BuiltinFirstNonEmpty

first_non_empty(value, value, ...) – returns the first value that is not empty. If all values are empty, then the empty value is returned. You can have as many values as you want.

lookup(val, pattern, field, pattern, field, ..., else_field)

class calibre.utils.formatter_functions.BuiltinLookup

lookup(val, pattern, field, pattern, field, ..., else_field) – like switch, except the arguments are field (metadata) names, not text. The value of the appropriate field will be fetched and used. Note that because composite columns are fields, you can use this function in one composite field to use the value of some other composite field. This is extremely useful when constructing variable save paths

switch(val, pattern, value, pattern, value, ..., else_value)

class calibre.utils.formatter_functions.BuiltinSwitch

switch(val, pattern, value, pattern, value, ..., else_value) – for each pattern, value pair, checks if the field matches the regular expression pattern and if so, returns that value. If no pattern matches, then else_value is returned. You can have as many pattern, value pairs as you want

List lookup

identifier_in_list(val, id, found_val, not_found_val)

class calibre.utils.formatter_functions.BuiltinIdentifierInList

identifier_in_list(val, id, found_val, not_found_val) – treat val as a list of identifiers separated by commas, comparing the string against each value in the list. An identifier has the format “identifier:value”. The id parameter should be either “id” or “id:regexp”. The first case matches if there is any identifier with that id. The second case matches if the regexp matches the identifier’s value. If there is a match, return found_val, otherwise return not_found_val.

in_list(val, separator, pattern, found_val, not_found_val)

class calibre.utils.formatter_functions.BuiltinInList

in_list(val, separator, pattern, found_val, not_found_val) – treat val as a list of items separated by separator, comparing the pattern against each value in the list. If the pattern matches a value, return found_val, otherwise return not_found_val.

list_item(val, index, separator)

class calibre.utils.formatter_functions.BuiltinListitem

list_item(val, index, separator) – interpret the value as a list of items separated by separator, returning the index`th item. The first item is number zero. The last item can be returned using `list_item(-1,separator). If the item is not in the list, then the empty value is returned. The separator has the same meaning as in the count function.

select(val, key)

class calibre.utils.formatter_functions.BuiltinSelect

select(val, key) – interpret the value as a comma-separated list of items, with the items being “id:value”. Find the pair with the id equal to key, and return the corresponding value.

str_in_list(val, separator, string, found_val, not_found_val)

class calibre.utils.formatter_functions.BuiltinStrInList

str_in_list(val, separator, string, found_val, not_found_val) – treat val as a list of items separated by separator, comparing the string against each value in the list. If the string matches a value, return found_val, otherwise return not_found_val. If the string contains separators, then it is also treated as a list and each value is checked.

List manipulation

count(val, separator)

class calibre.utils.formatter_functions.BuiltinCount

count(val, separator) – interprets the value as a list of items separated by separator, returning the number of items in the list. Most lists use a comma as the separator, but authors uses an ampersand. Examples: {tags:count(,)}, {authors:count(&)}

list_difference(list1, list2, separator)

class calibre.utils.formatter_functions.BuiltinListDifference

list_difference(list1, list2, separator) – return a list made by removing from list1 any item found in list2, using a case-insensitive compare. The items in list1 and list2 are separated by separator, as are the items in the returned list.

list_equals(list1, sep1, list2, sep2, yes_val, no_val)

class calibre.utils.formatter_functions.BuiltinListEquals

list_equals(list1, sep1, list2, sep2, yes_val, no_val) – return yes_val if list1 and list2 contain the same items, otherwise return no_val. The items are determined by splitting each list using the appropriate separator character (sep1 or sep2). The order of items in the lists is not relevant. The compare is case insensitive.

list_intersection(list1, list2, separator)

class calibre.utils.formatter_functions.BuiltinListIntersection

list_intersection(list1, list2, separator) – return a list made by removing from list1 any item not found in list2, using a case-insensitive compare. The items in list1 and list2 are separated by separator, as are the items in the returned list.

list_re(src_list, separator, search_re, opt_replace)

class calibre.utils.formatter_functions.BuiltinListRe

list_re(src_list, separator, search_re, opt_replace) – Construct a list by first separating src_list into items using the separator character. For each item in the list, check if it matches search_re. If it does, then add it to the list to be returned. If opt_replace is not the empty string, then apply the replacement before adding the item to the returned list.

list_sort(list, direction, separator)

class calibre.utils.formatter_functions.BuiltinListSort

list_sort(list, direction, separator) – return list sorted using a case-insensitive sort. If direction is zero, the list is sorted ascending, otherwise descending. The list items are separated by separator, as are the items in the returned list.

list_union(list1, list2, separator)

class calibre.utils.formatter_functions.BuiltinListUnion

list_union(list1, list2, separator) – return a list made by merging the items in list1 and list2, removing duplicate items using a case-insensitive compare. If items differ in case, the one in list1 is used. The items in list1 and list2 are separated by separator, as are the items in the returned list.

subitems(val, start_index, end_index)

class calibre.utils.formatter_functions.BuiltinSubitems

subitems(val, start_index, end_index) – This function is used to break apart lists of items such as genres. It interprets the value as a comma-separated list of items, where each item is a period-separated list. Returns a new list made by first finding all the period-separated items, then for each such item extracting the start_index to the end_index components, then combining the results back together. The first component in a period-separated list has an index of zero. If an index is negative, then it counts from the end of the list. As a special case, an end_index of zero is assumed to be the length of the list. Example using basic template mode and assuming a #genre value of “A.B.C”: {#genre:subitems(0,1)} returns “A”. {#genre:subitems(0,2)} returns “A.B”. {#genre:subitems(1,0)} returns “B.C”. Assuming a #genre value of “A.B.C, D.E.F”, {#genre:subitems(0,1)} returns “A, D”. {#genre:subitems(0,2)} returns “A.B, D.E”

sublist(val, start_index, end_index, separator)

class calibre.utils.formatter_functions.BuiltinSublist

sublist(val, start_index, end_index, separator) – interpret the value as a list of items separated by separator, returning a new list made from the start_index to the end_index item. The first item is number zero. If an index is negative, then it counts from the end of the list. As a special case, an end_index of zero is assumed to be the length of the list. Examples using basic template mode and assuming that the tags column (which is comma-separated) contains “A, B, C”: {tags:sublist(0,1,,)} returns “A”. {tags:sublist(-1,0,,)} returns “C”. {tags:sublist(0,-1,,)} returns “A, B”.

Other

assign(id, val)

class calibre.utils.formatter_functions.BuiltinAssign

assign(id, val) – assigns val to id, then returns val. id must be an identifier, not an expression

print(a, b, ...)

class calibre.utils.formatter_functions.BuiltinPrint

print(a, b, ...) – prints the arguments to standard output. Unless you start calibre from the command line (calibre-debug -g), the output will go to a black hole.

Recursion

eval(template)

class calibre.utils.formatter_functions.BuiltinEval

eval(template) – evaluates the template, passing the local variables (those ‘assign’ed to) instead of the book metadata. This permits using the template processor to construct complex results from local variables. Because the { and } characters are special, you must use [[ for the { character and ]] for the } character; they are converted automatically. Note also that prefixes and suffixes (the |prefix|suffix syntax) cannot be used in the argument to this function when using template program mode.

template(x)

class calibre.utils.formatter_functions.BuiltinTemplate

template(x) – evaluates x as a template. The evaluation is done in its own context, meaning that variables are not shared between the caller and the template evaluation. Because the { and } characters are special, you must use [[ for the { character and ]] for the } character; they are converted automatically. For example, template(‘[[title_sort]]’) will evaluate the template {title_sort} and return its value. Note also that prefixes and suffixes (the |prefix|suffix syntax) cannot be used in the argument to this function when using template program mode.

Relational

cmp(x, y, lt, eq, gt)

class calibre.utils.formatter_functions.BuiltinCmp

cmp(x, y, lt, eq, gt) – compares x and y after converting both to numbers. Returns lt if x < y. Returns eq if x == y. Otherwise returns gt.

strcmp(x, y, lt, eq, gt)

class calibre.utils.formatter_functions.BuiltinStrcmp

strcmp(x, y, lt, eq, gt) – does a case-insensitive comparison of x and y as strings. Returns lt if x < y. Returns eq if x == y. Otherwise returns gt.

String case changes

capitalize(val)

class calibre.utils.formatter_functions.BuiltinCapitalize

capitalize(val) – return value of the field capitalized

lowercase(val)

class calibre.utils.formatter_functions.BuiltinLowercase

lowercase(val) – return value of the field in lower case

titlecase(val)

class calibre.utils.formatter_functions.BuiltinTitlecase

titlecase(val) – return value of the field in title case

uppercase(val)

class calibre.utils.formatter_functions.BuiltinUppercase

uppercase(val) – return value of the field in upper case

String manipulation

re(val, pattern, replacement)

class calibre.utils.formatter_functions.BuiltinRe

re(val, pattern, replacement) – return the field after applying the regular expression. All instances of pattern are replaced with replacement. As in all of calibre, these are python-compatible regular expressions

shorten(val, left chars, middle text, right chars)

class calibre.utils.formatter_functions.BuiltinShorten

shorten(val, left chars, middle text, right chars) – Return a shortened version of the field, consisting of left chars characters from the beginning of the field, followed by middle text, followed by right chars characters from the end of the string. Left chars and right chars must be integers. For example, assume the title of the book is Ancient English Laws in the Times of Ivanhoe, and you want it to fit in a space of at most 15 characters. If you use {title:shorten(9,-,5)}, the result will be Ancient E-nhoe. If the field’s length is less than left chars + right chars + the length of middle text, then the field will be used intact. For example, the title The Dome would not be changed.

strcat(a, b, ...)

class calibre.utils.formatter_functions.BuiltinStrcat

strcat(a, b, ...) – can take any number of arguments. Returns a string formed by concatenating all the arguments

strcat_max(max, string1, prefix2, string2, ...)

class calibre.utils.formatter_functions.BuiltinStrcatMax

strcat_max(max, string1, prefix2, string2, ...) – Returns a string formed by concatenating the arguments. The returned value is initialized to string1. Prefix, string pairs are added to the end of the value as long as the resulting string length is less than max. String1 is returned even if string1 is longer than max. You can pass as many prefix, string pairs as you wish.

strlen(a)

class calibre.utils.formatter_functions.BuiltinStrlen

strlen(a) – Returns the length of the string passed as the argument

substr(str, start, end)

class calibre.utils.formatter_functions.BuiltinSubstr

substr(str, start, end) – returns the start’th through the end’th characters of str. The first character in str is the zero’th character. If end is negative, then it indicates that many characters counting from the right. If end is zero, then it indicates the last character. For example, substr(‘12345’, 1, 0) returns ‘2345’, and substr(‘12345’, 1, -1) returns ‘234’.

swap_around_comma(val)

class calibre.utils.formatter_functions.BuiltinSwapAroundComma

swap_around_comma(val) – given a value of the form “B, A”, return “A B”. This is most useful for converting names in LN, FN format to FN LN. If there is no comma, the function returns val unchanged

API of the Metadata objects

The python implementation of the template functions is passed in a Metadata object. Knowing it’s API is useful if you want to define your own template functions.

class calibre.ebooks.metadata.book.base.Metadata(title, authors=(u'Unknown', ), other=None, template_cache=None)

A class representing all the metadata for a book. The various standard metadata fields are available as attributes of this object. You can also stick arbitrary attributes onto this object.

Metadata from custom columns should be accessed via the get() method, passing in the lookup name for the column, for example: “#mytags”.

Use the is_null() method to test if a field is null.

This object also has functions to format fields into strings.

The list of standard metadata fields grows with time is in STANDARD_METADATA_FIELDS.

Please keep the method based API of this class to a minimum. Every method becomes a reserved field name.

is_null(field)

Return True if the value of field is null in this object. ‘null’ means it is unknown or evaluates to False. So a title of _(‘Unknown’) is null or a language of ‘und’ is null.

Be careful with numeric fields since this will return True for zero as well as None.

Also returns True if the field does not exist.

deepcopy()

Do not use this method unless you know what you are doing, if you want to create a simple clone of this object, use :method:`deepcopy_metadata` instead.

get_identifiers()

Return a copy of the identifiers dictionary. The dict is small, and the penalty for using a reference where a copy is needed is large. Also, we don’t want any manipulations of the returned dict to show up in the book.

set_identifiers(identifiers)

Set all identifiers. Note that if you previously set ISBN, calling this method will delete it.

set_identifier(typ, val)

If val is empty, deletes identifier of type typ

standard_field_keys()

return a list of all possible keys, even if this book doesn’t have them

custom_field_keys()

return a list of the custom fields in this book

all_field_keys()

All field keys known by this instance, even if their value is None

metadata_for_field(key)

return metadata describing a standard or custom field.

all_non_none_fields()

Return a dictionary containing all non-None metadata fields, including the custom ones.

get_standard_metadata(field, make_copy)

return field metadata from the field if it is there. Otherwise return None. field is the key name, not the label. Return a copy if requested, just in case the user wants to change values in the dict.

get_all_standard_metadata(make_copy)

return a dict containing all the standard field metadata associated with the book.

get_all_user_metadata(make_copy)

return a dict containing all the custom field metadata associated with the book.

get_user_metadata(field, make_copy)

return field metadata from the object if it is there. Otherwise return None. field is the key name, not the label. Return a copy if requested, just in case the user wants to change values in the dict.

set_all_user_metadata(metadata)

store custom field metadata into the object. Field is the key name not the label

set_user_metadata(field, metadata)

store custom field metadata for one column into the object. Field is the key name not the label

template_to_attribute(other, ops)

Takes a list [(src,dest), (src,dest)], evaluates the template in the context of other, then copies the result to self[dest]. This is on a best-efforts basis. Some assignments can make no sense.

smart_update(other, replace_metadata=False)

Merge the information in other into self. In case of conflicts, the information in other takes precedence, unless the information in other is NULL.

format_field(key, series_with_index=True)

Returns the tuple (display_name, formatted_value)

to_html()

A HTML representation of this object.

calibre.ebooks.metadata.book.base.STANDARD_METADATA_FIELDS

The set of standard metadata fields.

'''
All fields must have a NULL value represented as None for simple types,
an empty list/dictionary for complex types and (None, None) for cover_data
'''

SOCIAL_METADATA_FIELDS = frozenset([
    'tags',             # Ordered list
    'rating',           # A floating point number between 0 and 10
    'comments',         # A simple HTML enabled string
    'series',           # A simple string
    'series_index',     # A floating point number
    # Of the form { scheme1:value1, scheme2:value2}
    # For example: {'isbn':'123456789', 'doi':'xxxx', ... }
    'identifiers',
])

'''
The list of names that convert to identifiers when in get and set.
'''

TOP_LEVEL_IDENTIFIERS = frozenset([
    'isbn',
])

PUBLICATION_METADATA_FIELDS = frozenset([
    'title',            # title must never be None. Should be _('Unknown')
    # Pseudo field that can be set, but if not set is auto generated
    # from title and languages
    'title_sort',
    'authors',          # Ordered list. Must never be None, can be [_('Unknown')]
    'author_sort_map',  # Map of sort strings for each author
    # Pseudo field that can be set, but if not set is auto generated
    # from authors and languages
    'author_sort',
    'book_producer',
    'timestamp',        # Dates and times must be timezone aware
    'pubdate',
    'last_modified',
    'rights',
    # So far only known publication type is periodical:calibre
    # If None, means book
    'publication_type',
    'uuid',             # A UUID usually of type 4
    'languages',        # ordered list of languages in this publication
    'publisher',        # Simple string, no special semantics
    # Absolute path to image file encoded in filesystem_encoding
    'cover',
    # Of the form (format, data) where format is, for e.g. 'jpeg', 'png', 'gif'...
    'cover_data',
    # Either thumbnail data, or an object with the attribute
    # image_path which is the path to an image file, encoded
    # in filesystem_encoding
    'thumbnail',
    ])

BOOK_STRUCTURE_FIELDS = frozenset([
    # These are used by code, Null values are None.
    'toc', 'spine', 'guide', 'manifest',
    ])

USER_METADATA_FIELDS = frozenset([
    # A dict of dicts similar to field_metadata. Each field description dict
    # also contains a value field with the key #value#.
    'user_metadata',
])

DEVICE_METADATA_FIELDS = frozenset([
    'device_collections',   # Ordered list of strings
    'lpath',                # Unicode, / separated
    'size',                 # In bytes
    'mime',                 # Mimetype of the book file being represented

])

CALIBRE_METADATA_FIELDS = frozenset([
    'application_id',   # An application id, currently set to the db_id.
    'db_id',            # the calibre primary key of the item.
    'formats',          # list of formats (extensions) for this book
    # a dict of user category names, where the value is a list of item names
    # from the book that are in that category
    'user_categories',
    # a dict of author to an associated hyperlink
    'author_link_map',

    ]
)

ALL_METADATA_FIELDS =      SOCIAL_METADATA_FIELDS.union(
                           PUBLICATION_METADATA_FIELDS).union(
                           BOOK_STRUCTURE_FIELDS).union(
                           USER_METADATA_FIELDS).union(
                           DEVICE_METADATA_FIELDS).union(
                           CALIBRE_METADATA_FIELDS)

# All fields except custom fields
STANDARD_METADATA_FIELDS = SOCIAL_METADATA_FIELDS.union(
                           PUBLICATION_METADATA_FIELDS).union(
                           BOOK_STRUCTURE_FIELDS).union(
                           DEVICE_METADATA_FIELDS).union(
                           CALIBRE_METADATA_FIELDS)

# Metadata fields that smart update must do special processing to copy.
SC_FIELDS_NOT_COPIED =     frozenset(['title', 'title_sort', 'authors',
                                      'author_sort', 'author_sort_map',
                                      'cover_data', 'tags', 'languages',
                                      'identifiers'])

# Metadata fields that smart update should copy only if the source is not None
SC_FIELDS_COPY_NOT_NULL =  frozenset(['lpath', 'size', 'comments', 'thumbnail'])

# Metadata fields that smart update should copy without special handling
SC_COPYABLE_FIELDS =       SOCIAL_METADATA_FIELDS.union(
                           PUBLICATION_METADATA_FIELDS).union(
                           BOOK_STRUCTURE_FIELDS).union(
                           DEVICE_METADATA_FIELDS).union(
                           CALIBRE_METADATA_FIELDS) - \
                           SC_FIELDS_NOT_COPIED.union(
                           SC_FIELDS_COPY_NOT_NULL)

SERIALIZABLE_FIELDS =      SOCIAL_METADATA_FIELDS.union(
                           USER_METADATA_FIELDS).union(
                           PUBLICATION_METADATA_FIELDS).union(
                           CALIBRE_METADATA_FIELDS).union(
                           DEVICE_METADATA_FIELDS) - \
                           frozenset(['device_collections', 'formats',
                               'cover_data'])
                           # these are rebuilt when needed