Avid Media | Index Configuration Guide Version 2.
Legal Notices Product specifications are subject to change without notice and do not represent a commitment on the part of Avid Technology, Inc. This product is subject to the terms and conditions of a software license agreement provided with the software. The product may only be used in accordance with the license agreement. This product may be protected by one or more U.S. and non-U.S patents. Details are available at www.avid.com/patents. This guide is protected by copyright.
THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR AT&T MAKES ANY REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE. This product includes software developed by the University of California, Berkeley and its contributors. The following disclaimer is required by Paradigm Matrix: Portions of this software licensed from Paradigm Matrix.
Contents Using This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Symbols and Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 If You Need Help. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Avid Training Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Registering the Automation and Consolidate Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Registering a Provider with the Production Services Engine . . . . . . . . . . . . . . . . . . . . . . . . . 46 Creating a Consolidate Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Chapter 7 Configuring iNEWS for Media | Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Editing iNEWS Site Files. . . . . . .
Using This Guide This guide is intended for users of Avid MediaCentral Platform Services (MCS) with the Media | Index feature. This guide describes the configuration procedures for Media Index, such as configuring Media Index for single-server and cluster installations. For information on installing MCS, see the MediaCentral Platform Services Installation and Configuration Guide. For administrative information for MediaCentral | UX, see the Avid MediaCentral | UX Administration Guide.
If You Need Help If You Need Help If you are having trouble using your Avid product: 1. Retry the action, carefully following the instructions given for that task in this guide. It is especially important to check each step of your workflow. 2. Check the latest information that might have become available after the documentation was published. You should always check online for the most up-to-date release notes or ReadMe because the online version is updated whenever new information becomes available.
1 Media | Index Configuration Overview This document describes the tasks required to configure a Media Index system. Media Index comprises the following components: • Elasticsearch® service, which manages data storage and runs the search engine. • Elasticsearch tribe service, the federated client that enables multi-zone search functionality. • Data Import API, a bus-based service for metadata indexing, search, configuration, status, permissions, and updates for Media Index.
Checklist for Configuring Media | Index For more information on hardware requirements, see the MediaCentral Platform Services Hardware Guide. Media Index is installed with MediaCentral Platform Services by default, but the Media Index services are not started by default. Once configured, Media Index services start automatically when the server is powered-on. Media Index can be configured in both standalone and multi-zone environments.
Checklist for Configuring Media | Index Task Section Reference b If you are integrating with an Avid iNEWS system, “Configuring iNEWS for Media | Index” on configure the iNEWS servers for Media Index. b If you are integrating with an Interplay MAM system, configure the MAM servers for Media Index. 10 page 53. “Setting Up Interplay | MAM Sync Service” on page 59.
2 Configuring Media | Index for a Single Server When you set up Media Index, you enable Elasticsearch and several other services related to Media Index. Elasticsearch and Elasticsearch Tribe are used to index, store and search assets while the Media Index services add specific Avid logic to Elasticsearch. Elasticsearch indexes are split into “shards” and “replicas”. Each shard contains indexed data and each replica contains a copy of that data.
Configuring Media | Index 3. Delete the hostname value and type the value you want to use to assign to node. n If the node name variable is not set, Elasticsearch automatically generates node names when your node starts. If it is set to hostname, Elasticsearch uses the hostname of the node where the service is running. 4. Press the Escape key to return to command mode, then type :wq and press Return to write and quit the text editor. 5.
Configuring Media | Index The following table provides an example of which languages are used in searches within a multi-zone configuration with different languages enabled for different zones.
Validating the Media | Index Configuration n Hostnames can include only letters, numbers, dashes, and underscore symbols. - Elasticsearch Cluster Hosts (hostname or IP address of the local Media Central node) - Elasticsearch TCP Port - Elasticsearch HTTP Port - Elasticsearch Tribe TCP Port - Elasticsearch Tribe HTTP Port - Default Shards - Default Replicas 5. Click Apply. The system will display a message on the screen indicating “Update in progress...”.
Continuing the Installation The core services for your configuration must display a status of “Running,” and the system_data must display the shard count in green. Later, once you have configured an Interplay Production, MAM or iNEWS system, additional information on the indexes will appear in this window.
3 Configuring Media | Index for a Cluster When you set up Media Index, you enable Elasticsearch and several other services related to Media Index. Every node in an MCS cluster runs an Elasticsearch instance as well as an Elasticsearch-tribe instance. Elasticsearch and Elasticsearch Tribe are used to index, store and search assets while the Media Index services add specific Avid logic to Elasticsearch. Tribe instances allow you to perform read operations against the nodes in multiple zones.
Shard Count and Usage Configuring a Single Primary Shard Consider a MediaCentral cluster consisting of three nodes, as shown below: In this configuration, the MediaCentral UX System Settings were configured for: • 1 Default (primary) Shard • 2 Index Replicas Elasticsearch intelligently locates the shards across the three nodes. In this example the primary shard happens to be located on Node 1, but that might not always the case.
Shard Count and Usage might want to configure Media Index initially for six shards. As the additional nodes are added to the configuration, Elasticsearch relocates the primary shards to the new nodes so that the shards are evenly distributed across the six servers (one shard on each node). n Increasing the amount of shards without increasing the number of nodes on the system adds load to the system resources. For systems with spinning disks, there should ideally be nor more than two shards per node.
Preparing for the Media | Index Configuration Example 3: In this final example, MediaCentral has been configured for 3 primary shards and 1 replica: If Node 1 goes offline, Replica 1 on Node 3 becomes a primary shard and the index remains fully functional.
Adding Resources to the Cluster Adding Resources to the Cluster When you install a new MediaCentral system, all Media Index services are stopped except for the “avid-acs-media-index-configuration” service. After issuing the setup command, as detailed in the following process, additional services are started and resources are added to the cluster. To add resources to the cluster: 1.
Configuring Media | Index cluster hostname or site location (e.g. NewYork). In a multi-zone configuration, Avid recommends using the same value that you used for “Zone Name” during the multi-zone configuration process. Each zone in a multi-zone configuration requires a unique Media Index Cluster Name. c c Avid recommends that you not change the Cluster Name after your initial configuration. If you change the cluster name, Elasticsearch creates a new data folder on the next restart.
Configuring Media | Index Enabled Languages Language Selected for Search Languages in Results Zone 1: English, French English Zone 1: English Zone 2: English, Korean Zone 1: English, French Zone 2: English French Zone 1: French Zone 2: English, Korean Zone 1: English, French, Korean Zone 2: [no results] French Zone 1: French Zone 2: English, French, Korean c Zone 2: French If you change your language settings after you have configured Media Index, you must reset Media Index.
Validating the Media | Index Configuration 4. In the Supported Languages section, select the languages you want to make available for searching. You can select as many languages as needed, but Avid recommends you select only those languages your users are likely to use in a search. 5. Click the Default Language menu, and select the default language for indexed searches. 6.
Validating the Media | Index Configuration The core services for your configuration must display a status of “Running,” and the indexes must display the shard count in green. Note that the avid-acs-search-import service is installed on all cluster nodes, but is only ever active on one node. To monitor the fail count within the cluster: 1.
Continuing the Installation Continuing the Installation Depending upon your workflow, proceed to one or more of the following sections as applicable: • If you are installing Media Index in a multi-zone configuration, see “Setting Up Multi-Zone Search” on page 26. • If you are integrating with and Interplay Production or Interplay Archive system, you need to configure and start the Production Engine Bus Connector (PEBCo) service.
4 Setting Up Multi-Zone Search In a multi-zone MCS configuration, each zone runs its own Elasticsearch cluster with its own Elasticsearch Cluster Name. This is true for MCS single-servers as well as MCS servers in a Corosync cluster configuration. Single servers run a Elasticsearch cluster consisting of one node. These different Elasticsearch clusters are then connected to each other using the Elasticsearch tribe service.
Setting the Zone Bindings To update the Elasticsearch tribe configuration: 1. Using the Linux text editor, vi, open the Elasticsearch tribe configuration file for editing: vi /etc/elasticsearch-tribe/elasticsearch.yml 2. Add the following lines in the configuration file to add the binding information that allows the tribe to connect to another Media Index cluster: tribe..cluster.name: tribe..transport.tcp.
Setting the Zone Bindings Example from the master zone of the multi-zone configuration: [root@wavd-doc01 ~]# cat /etc/elasticsearch-tribe/elasticsearch.yml discovery.zen.ping.multicast.enabled: false tribe.local.cluster.name: Master_Zone tribe.local.transport.tcp.port: 9312 tribe.local.discovery.zen.ping.unicast.hosts: - "wavd-doc01:9300" http.port: 9201 transport.tcp.port: 9305 node.name: wavd-doc01-tribe tribe.Slave_Zone.cluster.name: Slave_Zone tribe.Slave_Zone.transport.tcp.port: 9313 tribe.Slave_Zone.
Setting the Zone Bindings http.port: 9201 transport.tcp.port: 9305 node.name: wavd-news-tribe tribe.Master_Zone.cluster.name: Master_Zone tribe.Master_Zone.transport.tcp.port: 9313 tribe.Master_Zone.discovery.zen.ping.unicast.hosts: - "wavd-doc01:9300" tribe.Slave_Zone.cluster.name: Slave_Zone tribe.Slave_Zone.transport.tcp.port: 9314 tribe.Slave_Zone.discovery.zen.ping.unicast.
Validating the Elasticsearch Configuration Validating the Elasticsearch Configuration When you finish updating the Elasticsearch configuration file, you can validate your setup by checking the Elasticsearch information for each server.
Validating the Elasticsearch Configuration 2 The import client listens for updates from the PEBco service. There is one instance of this service in each zone. In cluster configurations, the import client may or may not reside on the same server as the master. 3 Each remote zone tribe is bound to the local cluster. This this example, two remote zones are shown. Note that the tribe name reflects the local zone. 4 The local zone’s data node is listed.
Validating the Elasticsearch Configuration 5. Open a browser and navigate to http://:9201/_plugin/head, where sever_name is the host name or IP address of a server in a zone in your configuration — for example, the local zone — and the port number is configured for the tribe node of the zone. The Elasticsearch head plug-in Web page for the tribe node opens in your browser. 6. Verify that the tribe node lists the all indexes in your configuration as well as the local and remote data nodes.
5 Installing the Production Engine Bus Connector The Production Engine Bus Connector (PEBCo) queries the change table of the Interplay Production Engine and delivers that data to the Avid Common Services (ACS) bus where the import service takes these messages and sends the data to the Elasticsearch index. A timestamp is saved on the Interplay Production Engine, and the PEBCo service queries the Interplay Production Engine regularly and uses the timestamp to determine which assets have been already indexed.
Enabling Sync Events in the Interplay Administrator To enable the sync events (Media Index v2.3 or later): 1. Start Interplay Administrator and log on to the Interplay Production or Interplay Archive database that you want to be indexed. 2. In the Server section of the Interplay Administrator, click the Server Settings icon. The Server Settings view opens. 3. In the Update Tracking section, select Enabled to enable sync events. n You do not need to restart the Interplay Production or Archive Engine. 4.
Creating a PEBCo Instance on a Single MCS Server Creating a PEBCo Instance on a Single MCS Server The Production Engine Bus Connector (PEBco) is automatically installed during the MCS installation. Although before the service can be used, it must first be configured. To create a PEBCo instance: 1. Type the following command to create a PEBCo instance on the MCS server: pam-agent-ctrl add can be any meaningful name.
Creating a PEBCo Instance on an MCS Cluster n In a cluster, the addition of the cluster switch to this command adds the PEBco instance to all nodes simultaneously. A cluster configuration should return a response similar to the following: [root@wavd-mcs01 ~]# pam-agent-ctrl cluster add wavd-mcs Add resource: AvidPamAgent-wavd-mcs, Broker Host: 192.168.10.50 2.
Configuring the ACS Bus Service in Interplay Administrator Configuring the ACS Bus Service in Interplay Administrator The PEBCo configuration requires a series of bus calls to the MCS server. For the configuration to be successful, the Interplay Production and /or Interplay Archive Engines must have the proper bus URL so that the Interplay Administrator can configure the service correctly. To configure the ICS bus service: 1.
Configuring and Using the PEBco Service You configure the PEBCo service in the Production Engine Bus Connector view in the Interplay Administrator. This view has two tabs: • n - Fields that let you assign an instance of the PEBCo service to your Interplay Production or Interplay Archive database. - Buttons that let you start, stop, and resync the PEBCo service. - Information about the status of the instance which is assigned to the selected database.
Configuring and Using the PEBco Service c When you change the database properties, a re-index operation occurs to adapt the index structure to your change. This runs in the background but does not add data from these new properties to the existing data within the index. Only newly indexed assets contain the data from this property. To have this property available for all assets, you need to use the Resync button on the Manage/Status tab.
Configuring and Using the PEBco Service t n Click the Sync with Property Layout button to match these selections with those in the Interplay Administrator Property Layout view. Media Index base properties and common properties are selected by default, and you cannot deselect these properties. 3. Click Apply to save your selections and to make them available in MediaCentral UX. To configure an instance of the PEBCo service: 1.
6 Installing and Configuring Production Services for MediaCentral Delivery The Production Automation service and the Interplay Consolidate service are used for the MediaCentral Delivery feature.
Check List for Installing and Configuring Automation and Consolidate Services The following illustration shows the Production Services components and a Delivery workflow. MediaCentral Client (Browser) q Production Services Engine w e e w Media Services Automation r w Interplay Consolidate Interplay Delivery 1. The user initiates a Delivery request. 2. The Production Services Engine forwards the request to the Production Services Automation service, which determines the next step. 3.
Prerequisites for Installing and Configuring Automation and Consolidate Services Production Services Check List(Continued) Task Section Reference b Connect the Avid shared storage client to the Avid shared storage client documentation. System Director and mount the required workspaces. b Create at least one Interplay Consolidate profile. “Creating a Consolidate Profile” on page 50 No profile is needed for Production Services Automation.
Installing the Automation and Consolidate Services c When you create a Delivery profile for MediaCentral, use a network path to an Avid shared storage share for the Temporary Workspace on Sender parameter. For example, use \\avidstorage_server\Delivery_temp. This storage space is used for temporary media files created by the Interplay Consolidate service. If you use a local folder, the large amount of media created can cause system problems for the Delivery server.
Registering the Automation and Consolidate Services To install service descriptions: 1. Start the Production Services Engine and log in as administrator. 2. Click the Admin Tool button. The Interplay Production Services and Transfer Status tool opens. 3. Click the Services tab. The Services page displays the currently configured services. 4. Click Install/Upgrade. The Install/Upgrade Service dialog box opens. 5. Click the Browse button and navigate to the folder containing the service package (.
Registering a Provider with the Production Services Engine 8. Click Install/Upgrade. The service and its description appear on the Services page. The following illustration shows both services after they are registered. Registering a Provider with the Production Services Engine A service that is installed and registered is referred to as a provider. Starting with Interplay Production Services v3.1, some service providers operate as Windows services, and others operate as Windows applications.
Registering a Provider with the Production Services Engine 2. Substitute the hostname of the Production Services Engine. Following is an example of an Automation service .ini file with the default hostname “localhost”: #Mon May 05 11:34:54 EDT 2014 @5%?MaxJobs=1 @2%tMedia_Services_Engine_Hostname=localhost @1%tProvider_Name=a51-WG6_Consolidate_0123 Following is the same .ini file, edited to use the hostname “a51-MSE.
Registering a Provider with the Production Services Engine Registering and Connecting a Consolidate Provider Interplay Consolidate runs as a Windows application. You need to register the provider by connecting to the Production Services Engine. n If you try to connect to the Production Services Engine before the latest service is registered, the Status line in the Transcode Service dialog box reads: Error From Broker! UNKNOWN_SERVICE.
Registering a Provider with the Production Services Engine 3. Do the following: a. Provider Name — A default name for the provider is automatically supplied. Accept the default name or type a new name. In this example, the name is Consolidate_01. b. Production Services Engine Host Name — Type the name of the system running the Production Services Engine application. c. Automatically Connect — Select Yes to automatically connect the provider to the Production Services Engine when the application starts.
Creating a Consolidate Profile Creating a Consolidate Profile If users in your MediaCentral zone are going to perform MediaCentral Delivery operations that use “Deliver from Mark In to Mark Out,” an administrator must create at least one Consolidate profile. This default profile must be named “default_for_delivery.” The target video quality for this profile is displayed in the Deliver or Deliver To dialog box, and is used if the user selects “Deliver from Mark In to Mark Out.
Creating a Consolidate Profile Other options in the Consolidate profile, such as Handle and Interplay Checkin Folder, are also used for Deliver from Mark In to Mark Out operations. However, they are not shown in the Deliver To or Deliver to Me dialog box. To create a Consolidate profile: 1. Start the Production Services Engine. 2. Click the Admin Tool button. The Interplay Production Services and Transfer Status tool opens. 3. Click the Profiles tab. 4.
Creating a Consolidate Profile Consolidate Profile Settings(Continued) Parameter Description TargetAudioQuality Select an audio resolution for the consolidate operation: • None • MPEG1 Layer 2: Digital audio compressed to the MPEG1 Layer 2 specification at 96 Kb/s. • Uncompressed PCM: 16-bit 48-kHz digital audio • Uncompressed PCM 24-bit: 24-bit 48-kHz digital audio TargetVideoQuality Select a video resolution for the consolidate operation.
7 Configuring iNEWS for Media | Index When integrating iNEWS with Media Index, the iNEWS system relies on the sync agent in the Mediaindex utility program to push stories to Media Index. It must be configured to control what data gets indexed. To specify which Media Index the iNEWS system should use also requires configuration within iNEWS. This integration first appeared with the release of iNEWS v5.
iNEWS v5.2.x iNEWS v5.
Editing iNEWS Site Files Editing iNEWS Site Files The following site files must be edited on the iNEWS servers to enable Media Index functionality: • /site/dict/words • /site/dict/queues • /site/config The Mediaindex program must be pointed to the message broker by adding a definition in the /site/ dict/words file: • W_BINDBUS – Identifies the binding address for the MediaCentral platform’s bus server port, which is necessary for central indexing as part of iNEWS integration with Media Index.
Editing iNEWS Site Files To edit the iNEWS configuration file: 1. From the PuTTYCS application, select the PuTTY Filter created for sending server commands to all servers, such as iNEWS Consoles. c Always back up the /site/config file before making any changes. 2.
Manually Installing the log4cpp File For more information on the Q_MEDIA_INDEX token in /site/dict/queues, see Appendix C “Standard Dictionaries,” in the Avid iNEWS Setup and Configuration Guide. 3. Reconfigure the system. Manually Installing the log4cpp File The log4cpp library package for RHEL 6.5 is required for iNEWS to work with Media Index. It is necessary for the sync agent in the Mediaindex program to function with the Avid Common Services (ACS) bus.
Assigning the Media | Index Attribute Assigning the Media | Index Attribute Define the queues or folders in the iNEWS directory that will be monitored by Media Index by applying the Media Index attribute (+mi). To apply the Media Index attribute to a queue or folder: t From the console, type the dbtraits command in the following format: dbtraits +mi For example: dbtraits SHOW.
8 Setting Up Interplay | MAM Sync Service The following topics provide basic information to help you get started using Avid Interplay | MAM Sync Service Administrator: • About Interplay | MAM Sync Service Administrator • Opening Sync Service Administrator • Understanding the Sync Service Administrator Layout • Quitting Sync Service Administrator For more information on using the Interplay MAM Sync Service Administrator, see the Interplay | MAM Sync Service Administrator User’s Guide.
About Interplay | MAM Sync Service Administrator Note the following limitations: • All Attributes marked as “searchable” in the MAM data model are always sent to Central Index during synchronization; attributes marked as “not searchable” are not sent to Central Index and are not available for searching or in search-result lists. • Multi-value (MV) attributes and multi-value compound (MVC) attributes cannot be used for searching in Central Index and cannot be displayed in search-result lists.
About Interplay | MAM Sync Service Administrator Sync Service Administrator Configuration Options Some Sync Service Administrator features are subject to configuration. The default MAM installation comes with a full set of default configuration settings for Sync Service Administrator.
Opening Sync Service Administrator 7. Select SyncCentralIndex > Toggles and check if the value of the key EnableNewThesauriIds has the required value: n - True: Enables the new ID format for thesauri and legal lists. Recommended for new installations of Interplay MAM v5.4/MediaCental v2.5 or higher, or updating an installation with a small amount of data. Setting the value to “true” requires rebuilding the index for all assets. - False: This is the default value; allows the use of MAM v5.
Understanding the Sync Service Administrator Layout Understanding the Sync Service Administrator Layout Once you have logged in, Sync Service Administrator opens and displays the Monitor view. Sync Service Administrator includes the following areas and views: Area Function Title bar The Show mapping info link provides information on attribute mapping between Interplay MAM and Central Index.
Understanding the Sync Service Administrator Layout The Monitor View In most cases, your first experience with Sync Service Administrator will be in Monitor view, where the basic Sync Service Administrator features are close at hand. The Monitor view is arranged in six sections that provide information and controls to set up basic Sync Service Administrator functions. The following illustration and table describe the layout of the Monitor view.
Understanding the Sync Service Administrator Layout Item Function 3 Details Displays detailed information about the entry selected in the Recent events list. 4 Water mark indicator Indicates the load of the synchronization queue as a traffic light. Sync Index Administrator checks the number of high-priority entries in the DM_Sync table and shows a traffic light depending on water mark settings. You can update the water mark indicator manually by clicking the Refresh button.
Understanding the Sync Service Administrator Layout Item 1 Refresh controls Function Sync Service Administrator updates the display of recently synchronized objects every five seconds if the Auto refresh check box is checked. If you uncheck the Auto refresh check box you can update the display manually by clicking the Refresh button. 2 Synchronized objects list Displays a configurable number of recently synchronized objects (default is 3,000).
Understanding the Sync Service Administrator Layout The MAM / CI Compare View The MAM / CI Compare view provides statistics on the number of objects in MAM and Central Index per object class, and differences between them. Information is provided for object (asset) and EDL (sequence) classes. The information is displayed in an overview table, as shown in the following illustration. It shows the number of objects for each object and EDL class in MAM and Central Index.
Quitting Sync Service Administrator The Errors View The Errors view provides statistics on synchronization errors, and controls to reset or purge erroneous synchronization entries in Central Index.The following illustration and table describe the layout of the Errors view. Function 1 Refreshes the entries in the Object class table. 2 Displays the number of synchronization errors for each affected object class.
A Upgrading Media | Index The procedures for upgrading your MediaCentral system configured with Media Index depend on which version of MediaCentral Platform Services (MCS) you are upgrading from. For example, if you upgrade from v2.0 to v2.3, you must reset the Media Index indexes before the upgrade begins. If you upgrade from v2.2 to v2.3, you only need to migrate the Media Index schema data. Upgrading Media Index from v2.
Upgrading Media Index from v2.0.x or v2.1.x To reset Media Index to prepare for an upgrade: 1. If you are indexing an Interplay Production database, you must first determine the name of the PEBCo instance running on the MediaCentral server. Enter the following command on either a single MediaCentral server or any node in a cluster configuration: /opt/avid/bin/pam-agent-ctrl list The server returns the name of your PEBCo instance. 2.
Upgrading Media Index from v2.0.x or v2.1.x Editing the Elasticsearch-tribe Configuration File In multi-zone configurations, you also must edit the Elasticsearch-tribe configuration file in all zones to remove information about the zone or zones that you are upgrading. For example, in a three-zone configuration where you start by upgrading Zone_1, you must remove Zone_1’s information from the elasticsearch-tribe configuration file on Zone_2 and Zone_3.
Upgrading Media Index from v2.0.x or v2.1.x 6. In one of the remaining zones, use the Linux text editor, vi, to edit the elasticsearch.yml configuration file: vi /etc/elasticsearch-tribe/elasticsearch.yml 7. Delete the binding information for the zone that is being upgraded. The example below highlights the information for Zone_1 which is the zone being upgraded: [root@wavd-news ~]# vi /etc/elasticsearch-tribe/elasticsearch.yml discovery.zen.ping.multicast.enabled: false tribe.local.cluster.
Media | Index Backup Locations 2. Check to make sure the system_data index and index information are not listed. When Media Index is reset for your upgrade process, you can install the latest version of Media Central Platform Services. For more information, see one of the following topics: Once the MediaCentral upgrade is complete, you must reconfigure your Media Index configuration.
B Working with Media Index This section covers the additional commands and utilities that are used with systems configured for Media Index.
Displaying Information about Initialized Systems Displaying Information about Initialized Systems You can use the info command to display the following data for all available systems within all configured Elasticsearch clusters: • System id • System type • System display name • Number of assets To display system information, do the following: t Type the following command: /opt/avid/bin/avid-acs-elastic-index-setup -m info The following is an example of Media Index configured for an Avid Interpla
Checking the Current Index Status Checking the Current Index Status The Indexes pane in the MediaCentral UX System Settings can be used to view the status of the indexes in your configuration. n The Indexes pane displays only the indexes in the local zone. To display indexes of other zones, connect directly to the MediaCentral UX of that zone. To check the current status of your indexes: 1. Sign in to MediaCentral UX as an administrator. 2. Select System Settings from the Layout selector.
Using the Media | Index Restart Command Line Tool Using the Media | Index Restart Command Line Tool When Media Index is configured on a MediaCentral system, multiple new services are enabled. In the case of a cluster, many of these resources are added as resources to the Corosync / Pacemaker cluster. In an effort to simplify working with Media Index, Avid includes a utility that allows you to start, stop, or restart many of the services (or resources) in a single command.
Using the Media | Index Restart Command Line Tool n • crm — If working in a cluster configuration, the crm flag is required. For single-server configurations, this option is not required. • [command] — Specify one of the following commands to interact with the services: - start — start services - stop — stop services - restart — restart services - status — get the status To get command line tool help information, run the command tool without parameters.
Restarting the Search and PEBCo Services Restarting the Search and PEBCo Services If you find that the indexed search has stopped working in your MediaCentral environment, you can restart the search service on any node that has a problem and then resync your index. To check the status of the search service: 1. Sign in to MediaCentral UX as an administrator. 2. Select System Settings from the Layout selector. The System Settings layout opens. 3. In the Settings pane select Media Index > Cluster State.
Taking Nodes Offline in a MediaCentral Cluster 7. Click the Manage Status tab. 8. In the Manage section, click Resync to update the whole database content. Resyncing the database might take a long time to complete and invalidates the existing content of your index. During this process, the search only returns assets that are already processed within this run.
Removing Zones from a Multi-Zone Configuration • n Decrease number of shards. This option is only appropriate if a node will be taken offline permanently. You can change the replica count at any time, but changing the primary shard count requires you to reset the index, reconfigure it with the new shards count, and then re-index the data. This can be a time consuming process, especially for configurations with large indexes.
Removing Zones from a Multi-Zone Configuration 5. Restart the Elasticsearch tribe service on this zone by typing one of the following commands: t For single-server configurations: service elasticsearch-tribe restart t For cluster configurations: crm resource restart elasticsearchTribeEverywhere 6. In one of the remaining zones, use the Linux text editor, vi, to edit the elasticsearch.yml configuration file: vi /etc/elasticsearch-tribe/elasticsearch.yml 7.
Disabling Media Index Disabling Media Index Media Index components are automatically installed with the MediaCentral Platform Services software installation and cannot be removed. However, if you have configured Media Index on a MediaCentral single server or cluster and need to remove Media Index from the environment, the following processes details the required steps to delete the indexes and remove the Media Index configuration.
C Media | Index Custom Configuration The default setup described in this document covers most MCS configurations. In some instances, you might need to configure certain property files. n For more information on modifying the application properties file, see “Modifying application.properties” in the MediaCentral Platform Services Installation and Configuration Guide.
Modifying the Media | Index System Settings 3. In the Settings pane select Media Index > Indexes. The Media Index Indexes display in the Details pane. If you have indexed more than one system, such as an Interplay Production system and an iNEWS system, each system will be listed separately in the Details pane. Area or Component Description System Name (optional) The system name is set by the indexing agent.
Modifying the Media | Index System Settings Area or Component Description System ID The System ID is a unique identifier for an index that is generated by the indexing agent (PEBCO, MAM agent). It uniquely identifies the system in Media Index. Interplay Production systems are identified by a unique string of characters which cannot be altered. iNEWS systems use the system name as the System ID. Assets summary This area provides a brief summary of the assets indexed by Media Index.
Modifying the Media | Index System Settings 4. If you want to modify the name of the cluster, type the name in the Cluster Name field, and then click Apply. c If you change the cluster name, during a restart Elasticsearch creates a new data folder. If you had any indexes before you changed the cluster name, the indexes fail to appear in your configuration and the information in them is not accessible.
Modifying the Media | Index System Settings The Reset Media Index dialog box opens. 5. Type RESET to reset the system, and then click OK. You can cancel the action by clicking Cancel. The Media Index system is reset. 6. To restore the Elasticsearch tribe configuration, do the following: a. On each node in your configuration, use the Linux vi text editor to edit the backup copy of the Elasticsearch tribe configuration file: vi /etc/elasticsearch-tribe/elasticsearch.yml_backup b.
Adjusting Settings Through the ACS Monitor Adjusting Settings Through the ACS Monitor Many of the services used to create Media Index are governed by configuration files that contain parameters which can be adjusted to enhance or limit the operation of the system. While these files can be edited manually through Linux, Avid provides a user interface called the Avid ACS Monitor which can be used to adjust many of the same settings more easily.
Adjusting Settings Through the ACS Monitor Accessing the Avid ACS Monitor Tool The ACS Monitor is not enabled by default and must be manually started to interact with the tool and adjust system settings. To adjust the Media Index configuration through the ACS Monitor: 1. Log in to the MCS server as the Linux ‘root’ user. If you have a clustered configuration, log into the master node. 2. Start the avid-acs-monitor service: service avid-acs-monitor start 3.
Adjusting Settings Through the ACS Monitor If you click on any of the examples in this section, you will notice that they all share the same basic structure. Line 7 is the only line that is different between them. Rather than entering all the text manually, you can alter the contents of this example to update the maximum number of markers. 5.
Adjusting Settings Through the ACS Monitor 3. Locate the setConfigurationByField section and click the link for disableHighlighting. The following text appears in the right pane: { "serviceType": "avid.acs.media.index.configuration", "serviceRealm": "global", "serviceVersion": 0, "op": "setConfigurationByField", "paramSet": { "acs_disable_highlight": true } } 4. No adjustments are required, simply click the Query button in the right pane to save the change. 5.
Adjusting Settings Through the ACS Monitor If you click on any of the examples in this section, you will notice that they all share the same basic structure. Line 7 is the only line that is different between them. Rather than entering all the text manually, you can alter the contents of this example to alter the Media Index scalability options. 5.
Adjusting Settings Through the ACS Monitor 4. Locate the setBoostConfiguration item in the list of service options and click the link for setBoostConfiguration. The following text appears in the right pane: The paramSet lists three different categories or “entities”: asset (base), common, and tbmd (time based metadata). Each category is associated with various fields and boost values.
Adjusting Settings Through the ACS Monitor tbmd n category created creator duration id modified modifier startTC text thumbnailURL type Avid supports making changes to the asset, common, and tbmd categories. Creating additional categories is not supported. 5. Add or adjust the boost values as desired. In the following example, the “created” parameter has been added and assigned a boost values of 3; giving it a higher value than “category” and “creator”.
Adjusting Settings Through the ACS Monitor Disabling Search Suggestions When entering text in indexed search fields, search suggestions are displayed below the text box to assist users in finding the data they are looking for faster. While this is a useful feature, Media Index configurations that use large indexes might see a slow down in search performance when search suggestions are enabled. If desired, this feature can be disabled through the ACS Monitor tool. To disable the search suggestion data: 1.
Adjusting Settings Through the ACS Monitor To configure the search criteria behavior: 1. Follow the instructions for “Accessing the Avid ACS Monitor Tool” on page 90. 2. Enter “avid.acs.media.index.configuration” (without the quotes) in the search field at the top of the page to find the configuration service. 3. Expand the configuration service options by clicking the Service Operations List icon in the upper-left corner of the configuration service field. 4.
Setting Elasticsearch Memory Usage Setting Elasticsearch Memory Usage The Elasticsearch services require 4 GB of memory, which is set by default for standard configurations. If your configuration makes use of different memory requirements — for testing purposes — you can modify the Elasticsearch configuration files. To modify the Elasticsearch configuration files: 1.
Configuring the PEBCo for Custom Configurations Configuring the PEBCo for Custom Configurations For standard installations, you do not need to modify the properties file for the PEBCo. For custom configurations, you can edit the PAM agent services properties file. To configure the PEBCo: 1. Navigate to the directory containing the PAM agent services properties file: /opt/avid/etc/pam-agent-service 2. Using the Linux text editor, vi, open the pamagent-service.
Changing the Default Search Type Changing the Default Search Type By default, the Search bar and the Search pane in Media Central UX use the federated search. You can specify the default search type in MediaCentral UX by editing a properties file in your MediaCentral configuration. If you work with a cluster configuration, you must edit the application properties file on the master and slave nodes. Editing the file on load-balancing nodes is not required.
Changing Replica Counts Changing Replica Counts You can change the number of replicas for all indexes, for indexes at the specified alias, or for a specific index. You can view index names and any index alias names by using the Elasticsearch head plug-in Web page, which you can view by opening a browser and navigating to http://:9200/_plugin/head.
Changing the Index Refresh Interval Changing the Index Refresh Interval When you use Media Index to index an asset, you must wait for the next refresh for that asset to appear in a search. Refreshing the index requires some system resources, so by default the index refreshes after regular intervals instead of after each indexing operation. The interval is defined by the index.refresh_interval setting, which you can modify in either the Elasticsearch configuration file (/etc/elasticsearch/elasticsearch.
D Updating Media | Index Property Schemas Each Media Index index has a schema specific to its source system. You configure the schema in the Production Engine Bus Connector (PEBCo — for more information, see “Configuring and Using the PEBco Service” on page 37). If the list of fields in the PEBCo Canonical Data tab that you configure for indexing changes, then you must migrate the index to this new schema before the system can index new assets.
Property Changes in Production and Schema Updates Changing Properties Without Immediately Updating All Assets This approach adds properties to the index, but only new or updated assets carry the new properties. It performs a re-index so there is no interruption for the user. Index updates are blocked for all indexes until the re-indexing completes. After pressing Start, you might receive a timeout error after 10 seconds. This is expected, as the re-index operation needs more than 10 seconds to complete.
Property Changes in Production and Schema Updates Changing Properties and Restarting With an Empty Index You can use this method if you can accept that search remains incomplete until the initial indexing finishes. It saves the time needed for re-indexing, but depending on the number of assets in the database it can take quite some time for the initial indexing to finish.
Monitor the Re-index Process Monitor the Re-index Process You can monitor the re-index process in the Cluster State pane of the Media Index System Settings. During the re-index, the process creates a new, temporary index which gradually fills up with copies of the existing assets. Once the re-index is complete, Media Index deletes the original index and makes the new index active. By monitoring the number of documents in the existing and the new index, you can monitor the progress of the re-index process.
Monitor the Re-index Process Original index status When re-index starts, there is one additional column with the same basic name as the original index and a different ID at the end. The new index does not have the color bars that mark it as an active index. While the re-index progresses, the docs count slowly increases until it reaches the same number as the original index.
Monitor the Re-index Process The duration of the re-indexing process depends on a number of variables, such as the number of markers in the assets.
Avid Technical Support (USA) Product Information 75 Network Drive Burlington, MA 01803-2756 USA Visit the Online Support Center at www.avid.com/support For company and product information, visit us on the web at www.avid.