GDPR and IBM Connections / Docs On-Prem

At IBM Think back in March, the Connections Offering Management team promised us some information to help with GDPR compliency. The Connections Cloud information has been available for a while and the On-Premises docuemtation was on the way.

Today my friend and fellow IBM Champion Roberto alerted me to the technotes that are available.

IBM Connections: http://bit.ly/cnxGDPR

IBM Docs: http://bit.ly/docsGDPR

There is a new section in each of the knowledgebases entitled Managing personal data in accordance with GDPR – see here for Connections and here for Docs (note the docs section mostly points to the Connections KB).

The technotes refer to a number of standard tasks that should be configured as part of the install and configuration process – such as using SSL for HTTP traffic, disabling anonymous access etc.

I am sure that this information will evolve as time goes on, but its a great start.

Which version of IBM Docs to I have installed?

This morning, after a bit of attempting to work it out for myself .. I ask my good firiend and co speaker Roberto Boccadoro if there was a simple way to work out what version is installed?

WebSphere, Connections and the HTTP Server / Plugins etc all have command line options to find what version (and fixes) are installed and unfortunately its not so simple with docs ..

If you have kept the install files handy you can check but finding out the iFix version under the install\DocsApp directory where there is a handy file called ifix-version.json.

which gives you the iFix version .. which is just a number, 7, 15 or 25 etc. but how does that collate to which Docs version, CR and iFix you have installed?

Roberto has made a searchable list to make it simple to see what iFix version relates to which Docs version.

See Roberto’s awesome blog post here – and thank you so much my friend for doing this as it will save a bunch of time 🙂

IBM Docs 2 CR3 is here – and with some cool new features

So at last IBM Docs 2 CR3 has been released and with it some awesome new features that the community have been asking for (thank you IBM).

 

What’s new in Docs 2 CR3? LOADS of new features and fixes including …

The most exciting is probably the ability to deploy the Conversion server on linux (finally)!!

  • You can directly deploy Conversion server on Linux when installing IBM Docs.
  • If you already have IBM Docs installed with Conversion server on Windows, you can migrate existing Conversion server from Windows to Linux.

 

A track change feature for the Docs editor which is another feature that has been on the wish list for a while

  • Added Track Change support in Document Editor. People with editing access can use a new view in the sidebar to see what changes were made to the file and when and who. Changes can be filtered by the timeline and editors. File owners can turn Track Change on and off, and clear change history.

 

You can install CR3 straight over Docs 2, no need for the other CRs or iFixes – make sure you run the DB updates though to update the Docs schema.

 

Additional features and full realse notes can be found here: IBM Connections Docs 2 CR3 release notes

Download it now from Fix Central 🙂

Connections 6 migration observations

I am sure that lots of you have installed or started migrations to Connections 6 by now .. but I have found a couple of gotchas across the installs and updates that I have managed so far.

I am listing them here for sanities sake as I know when we google an error – this blog will show up 🙂

 

Connections 6 itself

 

Generally this was a nice smooth install and all worked as it should in a clean 6 environment

On testing a migration though I found issues when trying to migrate some of the DBs.

After dropping and restoring the DBs some were not updating to the latest DB2 version – this is resolved by running the update DB command

db2 upgrade db <DBNAME>

this upgrades the DB to the latest DB2 version – I run this after each restore command now to ensure the DB does update

for example

db2 restore db BLOGS from E:\install\V55_Backup\db2_backup taken at 20170718123342 ON E:\ into BLOGS
db2 upgrade db BLOGS

 

The metrics DB would not restore as it was complaining about heap size – after a quick google on the error I found a page suggesting just to change the application heap size using the following command

db2 UPDATE DB CFG FOR METRICS USING APPLHEAPSZ 4000

This resolved the problem and the metrics DB could be restored

 

I have also seen issues with the Connections DB update wizard when the databases have a lot of data. In a mature environment I generally run through the wizard and save the update commands to a text file. Then execute them manually from the db2 command window – for example

Activities
 1. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\activities\db2\upgrade-55CR2-60.sql
 2. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\activities\db2\appGrants.sql
 3. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\activities\db2\reorg.sql
 4. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\activities\db2\runstats.sql

Blogs
 1. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\blogs\db2\upgrade-55-60.sql
 2. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\blogs\db2\appGrants.sql
 3. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\blogs\db2\reorg.sql
 4. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\blogs\db2\runstats.sql

Bookmarks
 1. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\dogear\db2\upgrade-55-60.sql
 2. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\dogear\db2\appGrants.sql
 3. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\dogear\db2\reorg.sql
 4. E:\IBM\SQLLIB\bin\db2cmd -c -w -i db2 -td@ -vf connections.sql\dogear\db2\runstats.sql

.................. ETC.

For some reason if you have large DBs the wizard struggles massively. Running the commands manually from the command window also allows you to dump each commands logs out into a text file if you need to. Great for debugging and used a LOT when I ran through the AIX Oracle to Windows DB2 migration last year.

 

Once the DBs were restored and updated run the reorg and runstats against each of the DBs before you start it up and if you can also run the clearScheduler.sql for each DB.

That should sort out the majority of DB issues you may see during the update.

 

Connections TouchPoint

 

Again a fairly easy process once you work out what the documentation is talking about – things of note

The document speaks of copying the touchpoint folder contents to the htdocs directory –

 

 

 

 

 

 

you have to copy the entire folder – for example

Copy files from E:\install\Connections\Touchpoint\touchpoint to E:\IBM\HTTPServer\htdocs\touchpoint the wording in the documentation doesn’t really make that clear.

All of the paths etc in the documenation are case sensitive.

Another thing of note which has tripped me up twice now so I have to blog it (thank you Ben for the extra eyes on this one) is the contents of the touchpoint.deploy.properties

The touchpoint.deploy.properties which is included in the install files is different from the example the documentation gives, we basically mix the two and have the contents of the file looking like this

# when install to a clustered environment, provide clusterName.
# when install to a standalone server, provide node and server name.
clusterName=apps
#nodeName=icbvtDB2Node01
#serverName=server1

# setting required REE custom properties
ree.prop.image.upload.path=E:/IBM/CnxData/shared/touchpoint/upload_pic
ree.prop.profiles.app.entrypoint.host=connections.url.com
ree.prop.profiles.app.entrypoint.scheme=https
ree.prop.profiles.app.entrypoint.port=443

NOTE:  If using windows the image upload path must have / the linux way not \ the windows way. It will fail if you don’t even if you have it in “E:\IBM\etc…” it appears to be a java / python thing and the error will complain that it can not pass the properties file if the slashes are not /

 

IBM Docs CR2

 

To use IBM Docs with Connections 6 you must update to CR2 for Docs – that is fine (and see previous blog posts about issues with applying the CR2 fix).

There is an awesome gotcha that my good friend Roberto pointed me in the direction of the fix for – this one was a good one (thank you Marti for writing the blog)

The symptoms are Docs editing works fine, but when attempting to view a file the error : CLFAF400W: Can’t access the document repository appears

Its a super quick fix – edit the viewer-config.json which can be found in the (i.e \IBM\WebSphere\AppServer\profiles\Dmgr01\config\cells\<cell name>\IBMDocs-config)

Under the section

"components": [
 {
 "config": {
 "uploadRepository": "lcfiles",

find the line

 "class" : "com.ibm.concord.viewer.lc3.repository.LCFilesEJBRepository",

and replace it with

 "class": "com.ibm.concord.viewer.lc3.repository.LCFilesCMISRepository"

Just above that you can see the files_path

 "files_path": "E://IBM//CnxData//shared/files/upload"

Add the additional info to that section

 "j2c_alias": "connectionsAdmin", 
 "s2s_method": "j2c_alias", 
 "server_url": "https://connections.url.com/files",
 "files_path": "E://IBM//CnxData//shared/files/upload"

So the full section will now look like this:

 "components": [
 {
 "config": {
 "uploadRepository": "lcfiles", 
 "adapters": [
 {
 "config": {
 "j2c_alias": "connectionsAdmin", 
 "s2s_method": "j2c_alias", 
 "server_url": "https://connections.url.com/files",
 "files_path": "E://IBM//CnxData//shared/files/upload"
 }, 
 "id": "lcfiles", 
 "class": "com.ibm.concord.viewer.lc3.repository.LCFilesCMISRepository"
 },

 

Save the file and do a full sync once a restart of the viewer app is done the viewer will now work.

 

One other thing of note when it comes to migrating the Docs data and DBs is that almost every time you do a migration the wasadmin user will be out of sync and you will see duplicate user issues in the log.

easily resolved by running the syncMember command from the DMGR\bin directory

wsadmin.bat -lang jython

execfile("filesAdmin.py")

FilesMemberService.syncMemberExtIdByLogin("wasadmin")

 

Other things you should do once the docs data has been migrated is to run the generate thumbnails command (also from wsadmin in the filesAdmin section)

wsadmin.bat -lang jython

execfile("filesAdmin.py")

FilesThumbnailService.generateForAllFiles()

 

Also to migrate any drafts from the docs installer directory i.e

wsadmin.bat -lang jython -f E:/IBM/ConnectionsDocs/Docs/installer/docs/tasks/start_migration_tool.py

 

Hopefully you won’t come across any or more than one of these issues, but if you do hopefully there will be a quick resolution.

If I find anything else I will be sure to blog them.

Docs 2.0 CR2 – here we go again

I am blogging this mainly so I remember how to solve this

Big thanks to fellow ICS community member and all round great guy Robert Farstad for blogging this

Again you can see a problem when patching Docs 2 to CR2 as it can’t find the FQDN of the hosts.

The code has changed so it’s not possible to use the work around that we had previously.

There is a slight twist to getting it to work which Rob has explained fabulously so thank you mate .. you saved me a lot of grief .. and even IBM point people to that blog as they did when I opened a PMR to find out how to fix it .. 🙂

So Docs 2 CR1 and iFix 1 is not playing ball

I knew that some of our community had seen issues with applying Docs 2 CR1 and the accompanying iFix, so I purposely held off applying it.

But low and behold on Saturday afternoon when I did attempted to install CR1 I had problems.

The Docs and Viewer applications deployed without issues, but the Conversion server had these errors:

WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: "[remote_install_a_version, connections.hostname.com, [u\'connections.hostname.com\'], 8]"

WASX7017E: Exception received while running file "../execwas.py"; exception information: com.ibm.websphere.management.exception.AdminException: CWWSY0102E: Target with name connections.hostname.com was not found.

My friend and co-speaker Roberto Boccadoro had had this exact issue .. so off I went to his blog and round this post: IBM Connections Docs 2.0 CR1 fix does not install.

Excellent .. Followed the instructions but that didn’t resolve it – exactly the same issue. I checked the machine name, its host entry and FQDN and they were all correct – stumpted I asked in the Connections skype chat as I know other people had seen similar issues.

Monday morning .. I started getting some answers ….

Another quality participant Robert Farstad had seen issues with the Conversion application installing the iFix 1 for CR1 so I checked out his blog post here: Docs 2.0 CR1 iFix 001 – Conversion app will not update. Where there is an issue with the conversion-config.json in the Dmgr config not having the iFix attribute in the build section
i.e

 "ifix_version": 6,

I added this at version 1 for my test as I hadn’t applied any fixes at this point.

Tried to run the update again and you guessed it – failed again with the same issue as previously.

Roberto went back through his notes and found another post he had made on another Docs install he had issues applying the fix on ..IBM Connections Docs CR1 installation may fail with an error on Conversion server. This issue was exactly the same problem as I had the resolution is actually very simple

  1. Edit the applypatch.py in the Docs CR1 install directory
    Comment out the 2 lines in red from the def update_apps() section of the file

    def update_apps():
    
     “””update apps in present work directory”””
    
     for filename in os.listdir(‘./’):
    
     if(filename.endswith(“.ear”) or filename.endswith(“.ear.zip”)):
    
     appname = get_app_name(filename)
    
     CONFIG.call_was_task(‘update_app’,[appname, filename])
    
     #if os.path.exists(“docs_remote_installer.zip”):
    
     #update_conversion_binary()
  2. Under Docs CR1 install directory, go into the DocsApp subdirectory and rename the file concord-config.json to concord-config.json.OLD or something similar
  3. Re run the apply update script – it should work this time AND update the ifix version number in the config file to 6.
  4. Stop the Conversion cluster
  5. Under Docs CR1 install directory open the DocsConversion subdirectory. Unzip the docs_remote_installer.zip to a directory (in my case this was E:\Install\Docs\IBMConnectionsDocs_2.0_CR1\DocsConversion\docs_remote_installer)
  6. Check the [SYM_COUNT] which is the number of instances of symphony that are configured, typically this is between 4 and 8. Check the <Conversion_Install_Root>\symphony directory (in my case E:\IBM\ConnectionsDocs\Conversion\symphony)
  7. From the docs_remote_installer directory () run the command
    upgrade_node.bat --installroot <CONVERSION_INSTALL_ROOT> --symcount <SYM_COUNT>
    
    i.e
    
    upgrade_node.bat --installroot E:\IBM\ConnectionsDocs\Conversion --symcount 8
  8. Check fixpack.log in directory <CONVERSION_INSTALL_ROOT>\logs\
  9. Restart the Conversion cluster

Now all is good with my Docs update .. its working perfectly with Connections.

Thanks again to Roberto and Robert for helping me find the right solution.