This page will record any incidents and/or fixes required when the MDM subsystem fails.
Important :
when testing the Acceptance region, the MDM will be unavailable in Production and vice/versa
the MDM does not work well with multiple users doing simultaneous distributions
the MDM mailbox password is set to “never expire”, but auto updates every 80 days
Indications that the Process is down
After 15-20 minutes, the process start date fails to populate, attempt numbers stay at zero
2. Failure emails sent from
TC.DoNotReplyNAC-NepasrepondreCNA.TC@tc.gc.ca to TC.AAIR-RAINA.TC@tc.gc.ca
CAUSES
“Auto Discovery” and/or “SSL Certificate failure”
see PBI# 137134 fixed by SMGS Ticket #IM360871 (October 2021)
The MDM will not send out emails and the generic AAIR inbox begins receiving error emails (shown above)
This is a known recurrent error. Shared Services (SSC) needs to make an update on their end. Log a high-priority Orion ticket stating that the CAWIS-MDM is a “mission critical” application, when this happens.
An Auto-discovery incident also occurred in Dec 2021, but somehow self corrected without notifying SSC or Service Desk
2. PASSWORD or Authentication errors
See Pbi# 202037 fixed by Orion Ticket #INC33024 (September 2022)
Having established that the issue is not related to SSL Certificates or AutoDiscovery
Review the checklist at :
https://tcmarin.atlassian.net/wiki/spaces/CA/pages/1879802073
The MDM inbox is DNRENAC. The password is auto updated every 80 days, by the MDM process on the script server at \ncrws548\scripts\CivAv\CAWIS-MDM The log files in the “log” directory will tell you exactly when the process began failing
The refresh password process also updates the MDM - MAILBOX PASSWORD LAST CHECK DATE. If this date differs from the Password last modify date on TR47_CAWIS_GENERAL_PROPERTIES , it may indicate that the password was altered outside of the MDM process.
In this event, TC IM/IT can update the mailbox password
Enter the unencrypted password into the MDM - MAILBOX PASSWORD in CAWIS general properties
Null out the encryption value at MDM - MAILBOX PASSWORD SECRET. This needs to be done on the backend (Previous password and encrytion value are stored at MDM BACKUP - LAST KNOWN PASSWORD ,MDM BACKUP - LAST SECRET ENCRYPTION )
Change this value MDM - MAILBOX PASSWORD LAST CHECK DATE to SYSDATE + 2
The backlog of distribution jobs may take up to 15 minutes to begin executing afterward
The MDM inbox password will auto-update normally by way of the MDM process a day or so afterward. A new encrypted password and encryption key will then appear on the General properties table.
3. Hidden ASCII characters in an Airworthiness Directive (AD) subject
Very often the CAW Inspectors will simply copy and paste the subject from the AD document into the data entry page when processing a new AD.
A symptom of this would be that the MDM refuses to process a “specific” AD, but allows processing of others.
This issue was addressed in TFS Bug 13919
However, if it should reoccur for any reason, simply copy the English and French AD subjects into notepad, remove any strange looking characters, then paste those subject lines back into the page and resave .
4. MDM Email box disabled - or - password expired
Again this can only be fixed via an Orion ticket. The MDM password is in the the General properties table under the CAWIS code maintenance menu , if required.
(see SMGS TICKETS IM325231, IM307333 Solution - Resolution Code:Access Control U Had to reset and unlock the mailbox account for user ID DNRENAC)
(National Aircraft Certification – Distribution TC.CAWIS-SWIMN-DISTRIB-PROD.TC@TC.GC.CA)
5. CAWIS MDM OFFLINE/LOCKED
Distribution fails to Launch.
MDM runs 2 separate threads to send and process. The Sending thread exits without notifying the processing thread. The processing thread runs until hitting a “deadlock” counter (240 runs, approx 30 seconds = 2hrs) Afterward the processing thread kills the process and begins the distribution fresh
So this type of incident is self correcting. However if the process does not begin after 2hrs, then we need to investigate other reasons.
Add Comment