You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

projectdoc Toolbox

The Web API Extension for the projectdoc Toolbox can be employed to make bulk changes to documents. Since there is no safety net, these actions need to be take with care. This tip introduces a way to conduct this task.

Type
Level of Experience

Bulk changes on Confluence pages are usually conducted via the Confluence REST API. Using cURL, a bookmarklet, userscript, or any other form of REST client will do the trick. The Web API Extension for the projectdoc Toolbox provides additional REST services to access projectdoc documents and spaces. There are also services to access a particular property value or services to create documents from descriptors.

In case you use the REST API make sure that you do not delete valuable information. Usually you need recent backup and test your scripts on a copy of your production environment. The following tip shows how to lower the risk of unintentional data loss. A backup will be still required, but the changes of needing it are reduced. (smile)

Prerequisites

There are a number of option to access the Web API. For this tip two tools may be a valid choice, depending on using the change one time or multiple times. In case it is a one-shot, you could use the REST API Browser. In case you need it multiple times or consider to share the call, you could use cURL as part of a shell script.

Both approaches require the Web API Extension (available on the Atlassian Marketplace, version 6.2.0) for the projectdoc Toolbox

Get in action! In the first case I use the REST API Browser from Confluence. Please make sure you are using the latest version 3.2.3 (or later; see our blog post Resolved: REST API Browser showing API documentation again! for details).

Point your browser to: https://myserver.example.com/confluence/plugins/servlet/restbrowser#/resource/projectdoc-1-document

Uncheck "Show only public APIs" and insert "projectdoc" in the search box on the top left side.

Get in action! In the second case on a Linux system we prefer to use cURL (version 7.18, or later) and jq (version jq-1.5-1 or later).  curl is used in command lines or scripts to transfer data. jq is a lightweight and flexible command-line JSON processor. 

On a Linux machine's netrc configuration (~/.netrc), you may insert the following block with the credentials to access your Confluence server. This allows you to run the authentication without providing credentials on the command line.

machine www.example.com
login jane.doe
password XmysecretpasswordX

Protect your credentials!

 

Please make sure that the file is only readable and writable for you (-rw------- 1 jane.doe example 1356 Jan 21 08:44 /home/jane.doe/.netrc)

In the following steps this tip pretends that you have chosen this second option (smile).

Cautions Approach using the REST API

Since using the Web API does not provide a way to undo unintended changes, we provide a way to double check the implication of your actions.

In this example we show how to

  1. determine the documents you want to apply changes
  2. prepare the change
  3. apply the changes to one test document
  4. apply the changes to all documents


Determine the documents you want to apply changes

Get in action! In this example we are defining a query searching for documents of doctype Category containing a property named Type with value Type1. In the next step we will replace Type1 with the value Type2 for these documents.

The query
curl -s -n -G "https://www.smartics.eu/confluence-test/rest/projectdoc/1/document.json?select=Name,Type&from=MYEXAMPLE&max-hit-count=100&expand=property&max-result=100" \
	--data-urlencode "where=$<Doctype>=[category] AND $<Type>=[Type1]" | jq .

Short explanation:

  • The options -s -n -G stand for silent, use .netrc and use HTTP GET method
  • Followed by the URL to your Confluence server and the REST service for documents
  • The query path may contain all parameters that do not need to be URL-encoded directly to the URL, that is: the parameter values do not contain a reserved chars:
    ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" | "$" | ","
  • Each parameter that contains a value needed to be encoded use the option "--data-urlencode" (may be more than one)
    in the example above its just one: the where condition
  • Last but not least pipe "|" the result to jq to receive a formatted result (do not forget the punctuation mark "." followed after a whitespace following jq: "jq .")
Example Query Result
{
  "size": 2,
  "start-index": 0,
  "max-result": 2,
  "id-list": "128816586,128816588",
  "document": [
    {
      "id": 128816586,
      "property": [
        {
          "name": "Name",
          "value": "<ac:structured-macro ac:name=\"projectdoc-transclusion-parent-property\" ac:schema-version=\"1\" ac:macro-id=\"e8c27607-adfd-42e4-ace2-c2474ac3771c\"><ac:parameter ac:name=\"add-link\">false</ac:parameter><ac:parameter ac:name=\"property-name\">Name</ac:parameter></ac:structured-macro> / <ac:structured-macro ac:name=\"projectdoc-transclusion-property-display\" ac:schema-version=\"1\" ac:macro-id=\"8f80206a-419f-47cd-a099-da64c676a21d\"><ac:parameter ac:name=\"property-name\">Short Name</ac:parameter></ac:structured-macro>",
          "controls": "<br />"
        },
        {
          "name": "Type",
          "value": "<ac:structured-macro ac:name=\"projectdoc-name-list\"><ac:parameter ac:name=\"doctype\">category-type</ac:parameter><ac:parameter ac:name=\"render-no-hits-as-blank\">true</ac:parameter><ac:parameter ac:name=\"names\">Type1</ac:parameter><ac:parameter ac:name=\"property-restrict-value-range\">true</ac:parameter><ac:parameter ac:name=\"property\">Type</ac:parameter></ac:structured-macro>",
          "controls": ""
        }
      ],
      "section": []
    },
    {
      "id": 128816588,
      "property": [
        {
          "name": "Name",
          "value": "<ac:structured-macro ac:name=\"projectdoc-transclusion-parent-property\" ac:schema-version=\"1\" ac:macro-id=\"ac8cd350-faef-4367-8750-3739d5535286\"><ac:parameter ac:name=\"add-link\">false</ac:parameter><ac:parameter ac:name=\"property-name\">Name</ac:parameter></ac:structured-macro> / <ac:structured-macro ac:name=\"projectdoc-transclusion-property-display\" ac:schema-version=\"1\" ac:macro-id=\"471c11ea-4ab7-4fc4-96f8-8c9b95a264ea\"><ac:parameter ac:name=\"property-name\">Short Name</ac:parameter></ac:structured-macro>",
          "controls": "<br />"
        },
        {
          "name": "Type",
          "value": "<ac:structured-macro ac:name=\"projectdoc-name-list\"><ac:parameter ac:name=\"doctype\">category-type</ac:parameter><ac:parameter ac:name=\"render-no-hits-as-blank\">true</ac:parameter><ac:parameter ac:name=\"names\">Type1</ac:parameter><ac:parameter ac:name=\"property-restrict-value-range\">true</ac:parameter><ac:parameter ac:name=\"property\">Type</ac:parameter></ac:structured-macro>",
          "controls": ""
        }
      ],
      "section": []
    }
  ]
}

Explanation:

  • The result is starting with some general information about the size and the id-list, which we will need later
  • In this example the query has two results and for each result the two properties Name and Type are returned


Check! Check! Check!

 

(warning) Double and triple check the result set! (warning)

Make sure it is correct, as the changes we will make later on will be applied to the projectdoc documents in this result set!

Prepare the change

From the step above fetch the value of the JSON result property id-list.

In this example it is 128816586,128816588.

Get in action! Now we will prepare the new value for the property Type in a file "document.json"

document.json
{
      "property": [
        {
          "name": "Type",
          "value": "<ac:structured-macro ac:name=\"projectdoc-name-list\"><ac:parameter ac:name=\"doctype\">category-type</ac:parameter><ac:parameter ac:name=\"render-no-hits-as-blank\">true</ac:parameter><ac:parameter ac:name=\"names\">Type2</ac:parameter><ac:parameter ac:name=\"property-restrict-value-range\">true</ac:parameter><ac:parameter ac:name=\"property\">Type</ac:parameter><ac:parameter ac:name=\"\" /></ac:structured-macro>",
          "controls": "",
          "position": "replace"
        }
      ]
}

Explanation:

  • Here insert the whole Confluence macro as value, change the property value from Type1 to Type2 (the new value)
  • Confluence macros have a schema-version and a macro-id, it is recommended to remove them
  • Please remember to escape all quotes in the property value!


 

The Web API Extension provides other options to change the value of a property. Please read below the Information on More Options to Change the Values.

Apply the changes to one test document

We recommend to apply the changes now to one single test document. Preferable this step could be executed in a test environment.

Get in action! Start by preparing the HTTP query string

querystring=$(urlencode -m id-list=12345678)"&"$(urlencode -m new-version=true)"&"$(urlencode -m comment=Changed using the WEB-API)
echo $querystring

Explanation:

  • The code above is written for bash
  • The code is using the urlencode command provided by most Linux systems (/usr/bin/urlencode)
  • The echo command allows you to validate the query string

Get in action! Apply the change to the test page

curl -n --header "Content-Type: application/json" \
	-X PATCH "https://www.smartics.eu/confluence-test/rest/projectdoc/1/document.json?"${querystring} \
	--data-binary @document.json

The file "document.json" contains the information you want to apply to the documents. This is the document you created in step 3.

 

The name of the file in the file system must not contain the "@" symbol! It is only necessary for cURL.

 

(warning) Check the result in Confluence regarding the change in the test page.

Apply the changes to all documents

Now update and check the query string using all page IDs of the pages you intend to change.

Go back to Step 3 and copy the value of the property id-list.

Get in action! Insert the ids in the code below and check the new query string

querystring=$(urlencode -m id-list=128816586,128816588)"&"$(urlencode -m new-version=true)"&"$(urlencode -m comment=Changed using the WEB-API)
echo $querystring

Check the query string.

Get in action! Last step: Apply the change to all pages selected by their ids

 
  • This last line will apply the changes in the document document.json to all documents of id-list!
  • There is no rollback or undo!
  • The result will list for each page, whether it was a success or a failure.
  • There is no rollback when the change failed to alter one or some documents! The not failing documents will have the change applied to them.

Get in action! Execute curls code above:

curl -n --header "Content-Type: application/json" \
	-X PATCH "https://www.smartics.eu/confluence-test/rest/projectdoc/1/document.json?"${querystrng} \
	--data-binary @document.json
 
  • When some pages did not accept the change, but it succeeded for other pages: do not forget to remove those pages, which succeeded before you make a new attempt
  • The recommendation is this: Analyse the error, reproduce it in a test environment (e.g. by copying some of these pages) and start with a new query to collect only those pages which did not succeed and try again

More Options to Change the Values

The Web API Extension provides some more options to change properties as to replace the values completely with some new Confluence macro code, like we did in steps 4 and 5.

The documentation Documents Service / PATCH is describing those position property values, which can be used in the provided file document.json.

So instead of using replace as position, which expects, that the property to change contains the full Confluence macro code as property value, we recommend to check, if one of the other positions  e.g. delete-values, merge-values or replace-values provide a better usability for your use case.

  • With delete-values it is possible to delete a special property value
    In the example above we could have deleted the property value Type1 as a first step and then add the new value Type2 using merge-values
    delete-values just deletes the given value from the property, but keeps the macro information in place (even if it was the last value which has been deleted)
  • And merge-values checks if the new property value is already part of the property value; it does only add the value, if necessary


  • No labels