Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Document Properties Marker
doctypetopic
overridefalse


Short DescriptionThe Web API Extension for the projectdoc Toolbox can be employed to make bulk changes to documents. Since there is no safety net, these actions need to be take with care. This tip introduces a way to conduct this task.
Nameprojectdoc Bulk Changes to Documentshide
Short Name
Parent
Parent Property
property-nameName
hide
Audience
Name List
doctyperole
render-no-hits-as-blanktrue
property-restrict-value-rangetrue
propertyAudience
empty-as-nonetrue

Subject
Name List
doctypesubject
propertySubject

Categories
Name List
doctypecategory
property-restrict-value-rangetrue
propertyCategories

Tags
Tag List
propertyTags

Flagshide
Iteration

Iteration
valuefilled

hide
Type

Name List
doctypetopic-type
render-no-hits-as-blanktrue
namesTip
property-restrict-value-rangetrue
propertyType


Level of Experience

Name List
doctypeexperience-level
render-no-hits-as-blanktrue
namesExpert
propertyLevel of Experience


Expected Duration1h
Sponsors
Name List
doctypestakeholder
render-no-hits-as-blanktrue
propertySponsors

Sort Keyhide



Section
show-titlefalse
titleDescription

Bulk changes on Confluence pages are usually conducted via the Confluence REST API. Using cURL, a bookmarklet, userscript, or any other form of REST client will do the trick. The Web API Extension for the projectdoc Toolbox provides additional REST services to access projectdoc documents and spaces. There are also services to access a particular property value or services to create documents from descriptors.

In case you use the REST API make sure that you do not delete valuable information. Usually you need recent backup and test your scripts on a copy of your production environment. The following tip shows how to lower the risk of unintentional data loss. A backup will be still required, but the changes of needing it are reduced. (smile)

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
In this Tip you will learn by an example, how to define a search to find the documents you want to bulk change, define this change, apply it to a test document and finally do the real change.

In our example we assume, that some documents of Doctype Category which all have a property Type with value Type1 already exist.

The goal is to change all these documents to have value Type2 for the property Type, instead of having the value Type1.


HTML
<style>
div.pdsectionindent {
    margin-top: 20px;
}
projectdoc-has-id-or-tags projectdoc-steps step step-level-1 div.projectdoc-steps div.step-level-1, div.projectdoc-steps div.step-level-1:first-child {
	font-size: 20px;
}

div.pdsectionindent p, div.pdsectionindent pre, div.pdsectionindent ul, div.pdsectionindent div.code, div.pdsectionindent div.projectdoc-information-macro {
    margin-left: 40px;
}
</style>


Section
titlePrerequisites

There are a number of options to access the Web API. For this tip two tools may be a valid choice, depending on using the change one time or multiple times. In case it is a one-shot, you could use the REST API Browser. In case you need it multiple times or consider to share the call, you could use cURL as part of a shell script.

Both approaches require the Web API Extension (available on the Atlassian Marketplace, version 6.2.0) for the projectdoc Toolbox

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Choosing the first option, using the REST API Browser from Confluence, please make sure you are using the latest version 3.2.3 (or later; see our blog post Resolved: REST API Browser showing API documentation again! for details).

Point your browser to: https://myserver.example.com/confluence/plugins/servlet/restbrowser#/resource/projectdoc-1-document

Uncheck "Show only public APIs" and insert "projectdoc" in the search box on the top left side to see all projectdoc REST-API calls we provide.

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Choosing the seconf option prefer to use cURL (version 7.18, or later) and jq (version jq-1.5-1 or later).  cURL is used in command lines or scripts to transfer data. jq is a lightweight and flexible command-line JSON processor. 

Using cURL in combination with netrc is a very easy and quite secure way to handle your credentials for Confluence. The following netrc configuration (~/.netrc) for linux systems allows to run the authentication without providing credentials on the command line (for more information and other options we recomend the tip: REST Login to Confluence with cURL).

machine www.example.com
login jane.doe
password XmysecretpasswordX

Caution Box
titleProtect your credentials!

.netrc file should only be readable and writable for you (-rw------- 1 jane.doe example 1356 Jan 21 08:44 /home/jane.doe/.netrc)

In the following steps this tip pretends that you have chosen the second option using cURL(smile).


Section
titleCautious Approach using the REST API

Since using the Web API does not provide a way to undo unintended changes, we provide a way to double check the implication of your actions.

In this example we show how to

  1. determine the documents you want to apply changes
  2. prepare the change
  3. apply the changes to one test document
  4. apply the changes to all documents


Steps


Section
csspdsectionindent
idDetermineTheDocumentsYouWantToApplyChanges
titleDetermine the documents you want to apply changes

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Let us start by defining a query searching for documents of Doctype Category containing a property named Type with value Type1.

Code Block
languagebash
themeMidnight
titleThe query
curl -s -n -G "https://www.example.com/confluence/rest/projectdoc/1/document.json?select=Name,Type&from=MYEXAMPLE&max-hit-count=100&expand=property&max-result=100" \
	--data-urlencode "where=$<Doctype>=[category] AND $<Type>=[Type1]" | jq .

Short explanation:

  • The options -s -n -G stand for silent, use .netrc and use HTTP GET method
  • Followed by the URL to your Confluence server and the REST service for documents
  • The query path may contain all parameters that do not need to be URL-encoded directly to the URL, that is: the parameter values do not contain a reserved chars:
    ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" | "$" | ","
  • Each parameter that contains a value needed to be encoded use the option "--data-urlencode" (may be more than one)
    in the example above its just one: the where condition (read here more on Search Tips)
  • Last but not least pipe "|" the result to jq to receive a formatted result
    (the punctuation mark "." followed after a whitespace following jq: "jq ." is mandatory!)
Code Block
languageactionscript3
titleExample Query Result
linenumberstrue
collapsetrue
{
  "size": 2,
  "start-index": 0,
  "max-result": 2,
  "id-list": "128816586,128816588",
  "document": [
    {
      "id": 128816586,
      "property": [
        {
          "name": "Name",
          "value": "<ac:structured-macro ac:name=\"projectdoc-transclusion-parent-property\" ac:schema-version=\"1\" ac:macro-id=\"e8c27607-adfd-42e4-ace2-c2474ac3771c\"><ac:parameter ac:name=\"add-link\">false</ac:parameter><ac:parameter ac:name=\"property-name\">Name</ac:parameter></ac:structured-macro> / <ac:structured-macro ac:name=\"projectdoc-transclusion-property-display\" ac:schema-version=\"1\" ac:macro-id=\"8f80206a-419f-47cd-a099-da64c676a21d\"><ac:parameter ac:name=\"property-name\">Short Name</ac:parameter></ac:structured-macro>",
          "controls": "<br />"
        },
        {
          "name": "Type",
          "value": "<ac:structured-macro ac:name=\"projectdoc-name-list\"><ac:parameter ac:name=\"doctype\">category-type</ac:parameter><ac:parameter ac:name=\"render-no-hits-as-blank\">true</ac:parameter><ac:parameter ac:name=\"names\">Type1</ac:parameter><ac:parameter ac:name=\"property-restrict-value-range\">true</ac:parameter><ac:parameter ac:name=\"property\">Type</ac:parameter></ac:structured-macro>",
          "controls": ""
        }
      ],
      "section": []
    },
    {
      "id": 128816588,
      "property": [
        {
          "name": "Name",
          "value": "<ac:structured-macro ac:name=\"projectdoc-transclusion-parent-property\" ac:schema-version=\"1\" ac:macro-id=\"ac8cd350-faef-4367-8750-3739d5535286\"><ac:parameter ac:name=\"add-link\">false</ac:parameter><ac:parameter ac:name=\"property-name\">Name</ac:parameter></ac:structured-macro> / <ac:structured-macro ac:name=\"projectdoc-transclusion-property-display\" ac:schema-version=\"1\" ac:macro-id=\"471c11ea-4ab7-4fc4-96f8-8c9b95a264ea\"><ac:parameter ac:name=\"property-name\">Short Name</ac:parameter></ac:structured-macro>",
          "controls": "<br />"
        },
        {
          "name": "Type",
          "value": "<ac:structured-macro ac:name=\"projectdoc-name-list\"><ac:parameter ac:name=\"doctype\">category-type</ac:parameter><ac:parameter ac:name=\"render-no-hits-as-blank\">true</ac:parameter><ac:parameter ac:name=\"names\">Type1</ac:parameter><ac:parameter ac:name=\"property-restrict-value-range\">true</ac:parameter><ac:parameter ac:name=\"property\">Type</ac:parameter></ac:structured-macro>",
          "controls": ""
        }
      ],
      "section": []
    }
  ]
}

Explanation:

  • The result is starting with some general information about the size and the id-list, which we will need later
  • In this example the query has two documents with id 128816586 and id 128816588 as query result and for each document two properties Name and Type are returned


Caution Box
titleCheck! Check! Check!

(warning) Double and triple check the result set! (warning)

Make sure it is correct, as the changes in step 4 will be applied to the documents in this result set!



Section
csspdsectionindent
idPrepareTheChange
titlePrepare the change

From the step above fetch the value of the JSON result property id-list for later usage.

In this example it is 128816586,128816588.

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Now prepare the new value for the property Type in a file "document.json" (read more on changing documents →  Documents Service / PATCH)

Code Block
titledocument.json
{
      "property": [
        {
          "name": "Type",
          "value": "<ac:structured-macro ac:name=\"projectdoc-name-list\"><ac:parameter ac:name=\"doctype\">category-type</ac:parameter><ac:parameter ac:name=\"render-no-hits-as-blank\">true</ac:parameter><ac:parameter ac:name=\"names\">Type2</ac:parameter><ac:parameter ac:name=\"property-restrict-value-range\">true</ac:parameter><ac:parameter ac:name=\"property\">Type</ac:parameter><ac:parameter ac:name=\"\" /></ac:structured-macro>",
          "controls": "",
          "position": "replace"
        }
      ]
}

Explanation:

  • Insert the whole Confluence macro as value for the property and change the property value from Type1 to Type2 (the new value)
  • Confluence macros have a schema-version and a macro-id, it is recommended to remove them
  • Please remember to escape all quotes in the property value!


Tip Box

The Web API Extension provides other options to change the value of a property. Please read below the Information on More Options to Change the Values.



Section
csspdsectionindent
idApplyTheChangesToOneTestDocument
titleApply the changes to one test document

We recommend to apply the changes now to one single test document. Preferable this step could be executed in a test environment.

So create a text document, fetch its page id e.g. by viewing the Page Information and copying the pageid from the URL part (e.g. confluence/pages/viewpage.action?pageId=43647697) or use Copy Page ID (one of our Bookmarklets).

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Start by preparing the HTTP query string

Code Block
languagebash
themeMidnight
querystring=$(urlencode -m id-list=12345678)"&"$(urlencode -m new-version=true)"&"$(urlencode -m comment=Changed using the WEB-API)
echo $querystring

Explanation:

  • The code above is written for bash
  • The code is using the urlencode command provided by most Linux systems (/usr/bin/urlencode)
  • The $() expressions start a subshell and evaluates urlencode command inline
  • Fill in the page id of the Confluence test page as value of id-list
  • Keep all spaces and do not add aditional spaces besides the spaces in the comment value
  • The echo command allows you to validate the query string

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Apply the change to the test page

Code Block
languagebash
themeMidnight
curl -n --header "Content-Type: application/json" \
	-X PATCH "https://www.example.com/confluence/rest/projectdoc/1/document.json?"${querystring} \
	--data-binary @document.json

Explanation:

  • Change the URL to your Confluence server
  • The file "document.json" contains the change to apply to the documents. This is the document you created in step 3


Tip Box

The name of the file in the file system must not contain the "@" symbol! It is only necessary for cURL.


Caution Box

(warning) Check the result in Confluence regarding the change in the test page.



Section
csspdsectionindent
idApplyTheChangesToAllDocuments
titleApply the changes to all documents

Now update and check the query string using all page Ids of the pages you intend to change.

Go back to Step 3 and copy the value of the property id-list.

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Insert the ids in the code below and check the new query string

Code Block
languagebash
themeMidnight
querystring=$(urlencode -m id-list=128816586,128816588)"&"$(urlencode -m new-version=true)"&"$(urlencode -m comment=Changed using the WEB-API)
echo $querystring

Check the query string.

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Last step: Apply the change to all pages selected by their ids

Caution Box
  • This last line will apply the changes in the document document.json to all documents of id-list!
  • There is no rollback or undo!
  • The result will list for each page, whether it was a success or a failure.
  • There is no rollback when the change failed to alter one or some documents! The not failing documents will have the change applied to them.

HTML Span
span<span class="aui-icon aui-icon-small aui-iconfont-arrow-right">Get in action!</span>
 Execute curl code:

Code Block
languagebash
themeMidnight
curl -n --header "Content-Type: application/json" \
	-X PATCH "https://www.smartics.eu/confluence-test/rest/projectdoc/1/document.json?"${querystrng} \
	--data-binary @document.json


Tip Box
  • When some pages did not accept the change, but it succeeded for other pages: do not forget to remove those pages, which succeeded before you make a new attempt
  • The recommendation is this: Analyse the error, reproduce it in a test environment (e.g. by copying some of these pages) and start with a new query to collect only those pages which did not succeed and try again





Section
titleMore Options to Change the Values

The Web API Extension provides some more options to change properties as to replace the values completely with some new Confluence macro code, like we did in steps 4 and 5.

The documentation Documents Service / PATCH is describing those position property values, which can be used in the provided file document.json.

So instead of using replace as position, which expects, that the property to change contains the full Confluence macro code as property value, we recommend to check, if one of the other positions  e.g. delete-values, merge-values or replace-values provide a better usability for your use case.

  • With delete-values it is possible to delete a special property value
    In the example above we could have deleted the property value Type1 as a first step and then add the new value Type2 using merge-values
    delete-values just deletes the given value from the property, but keeps the macro information in place (even if it was the last value which has been deleted)
  • And merge-values checks if the new property value is already part of the property value; it does only add the value, if necessary


Section
titleConclusion

In this Tip you learned how to bulk change a property value of multiple projectdoc Documents by fulfilling 4 easy steps:

  1. determine the documents you want to apply changes
  2. prepare the change
  3. apply the changes to one test document
  4. apply the changes to all documents


Section
ignore-template-buttonstrue
titleSubordinate Topics


Hide From Reader

Create from template
blueprintModuleCompleteKeyde.smartics.atlassian.confluence.smartics-projectdoc-confluence-space-core:projectdoc-blueprint-doctype-topic
buttonLabelCreate Topic

Display Table
doctypetopic
render-no-hits-as-blanktrue
render-modedefinition
selectName, Short Description
restrict-to-immediate-childrentrue
sort-bySort Key, Name


Section
required-permissionswrite-access
titleNotes


Section
titleReferences


Tour
render-no-hits-as-blanktrue
render-as-definition-listtrue
marker-column-property-nameTitle
replace-title-with-nametrue


TitleShort Description





Section
titleResources


Tour
render-no-hits-as-blanktrue
render-as-definition-listtrue
marker-column-property-nameTitle
replace-title-with-nametrue


TitleShort Description
Web API Extension
projectdoc ToolboxEnhance your Documentation Experience ! The projectdoc Toolbox supports agile teams in writing documentation collaboratively
cURLCommand line tool and library for transferring data with URLs
REST API Browser from ConfluenceInteractive API explorer for Atlassian products
REST Login to Confluence with cURL
 Search Tips
Documents Service / PATCH
Copy Page ID




Piwik Set Multiple Custom Variables


NameValue
Departmentprojectdoc
Categoryprojectdoc-tip
Typehowto