Skip to content

Commit

Permalink
Merge pull request #100 from rmgrimm/fixes
Browse files Browse the repository at this point in the history
Fix uploads; add exception classes; other fixes
  • Loading branch information
rmgrimm authored Nov 8, 2023
2 parents c71e6bd + cf5b30f commit 8932df1
Show file tree
Hide file tree
Showing 42 changed files with 845 additions and 527 deletions.
31 changes: 18 additions & 13 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
= 3scale CMS Tools

:sectnums:
:toc:

== Overview

The 3scale CMS tools project provides the ability to interact with 3scale's
Content Management System programmatically. The project also
demonstrates API-first code generation.
Expand All @@ -24,21 +25,24 @@ This repository provides the following artifacts:

=== Command-Line Interface

The 3scale CMS tools project provides a CLI tool that provides a convenient
mechanism for interacting with the 3scale Content Management API. It is
implemented using link:https://quarkus.io[Quarkus] and
The 3scale CMS tools project includes a Command-Line Interface (CLI) tool that
provides a convenient mechanism for interacting with the 3scale Developer Portal
API. It is implemented using link:https://quarkus.io[Quarkus] and
link:https://picocli.info[picocli].

More information on CLI Usage may be found in the
link:docs/cli-usage.adoc[Command-Line Interface Usage] document.

=== Container Images

Images are available for running the command-line interface in container
environments. Images are available in the GitHub Packages repository at
link:https://github.com/orgs/FwMotion/packages?repo_name=3scale-cms[ghcr.io/fwmotion/3scale-cms].
link:https://github.com/FwMotion/3scale-cms/pkgs/container/3scale-cms[ghcr.io/fwmotion/3scale-cms].

The following image tags are used:

* **VERSION** _(eg, 1.0.0)_: Execution of the CLI using the JVM
* **VERSION-native** _(eg, 1.0.0-native)_: Execution of a native-built CLI
* **VERSION** _(eg, v2.0.1)_: Execution of the CLI using the JVM
* **VERSION-native** _(eg, v2.0.1-native)_: Execution of a native-built CLI
* **latest**: The latest version available using JVM-mode builds
* **latest-native**: The latest version available using native-mode builds

Expand Down Expand Up @@ -83,12 +87,13 @@ the available endpoints for managing CMS objects, along with the "provider"
endpoint of the Account Management API. This provider endpoint is used for
lookup of the Developer Portal base URL and the Developer Portal access code.

=== REST Client JAR
=== REST Client JARs

The 3scale CMS tools project provides a standalone REST client JAR. The JAR is
available in the GitHub Packages maven repository.
The 3scale CMS tools project provides JARs to handle interaction with 3scale's
Developer Portal API. These JARs are available in the GitHub Packages maven
repository.

To use the JAR in a Maven project, first include the following repository
To use a JAR in a Maven project, first include the following repository
definition:

[source,xml]
Expand All @@ -110,8 +115,8 @@ definition:
</repositories>
----

To include the client JAR as a dependency, use the following dependency
definition:
As an example, to include the REST client JAR as a dependency, use the following
dependency definition:

[source,xml]
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import com.fwmotion.threescale.cms.cli.support.CmsObjectPathKeyGenerator;
import com.fwmotion.threescale.cms.cli.support.CmsSectionToTopComparator;
import com.fwmotion.threescale.cms.cli.support.PathRecursionSupport;
import com.fwmotion.threescale.cms.exception.ThreescaleCmsCannotDeleteBuiltinException;
import com.fwmotion.threescale.cms.model.CmsObject;
import io.quarkus.logging.Log;
import jakarta.annotation.Nonnull;
Expand Down Expand Up @@ -115,6 +116,8 @@ static void deleteObjects(ThreescaleCmsClient client, CmsObjectPathKeyGenerator
Log.info("Deleting " + objectName);
try {
client.delete(object);
} catch (ThreescaleCmsCannotDeleteBuiltinException e) {
Log.info("Could not delete built-in " + objectName);
} catch (Exception e) {
// TODO: Handle exceptions properly, and provide better logs
// indicating the reason for failure
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
import com.fwmotion.threescale.cms.ThreescaleCmsClient;
import com.fwmotion.threescale.cms.cli.support.*;
import com.fwmotion.threescale.cms.model.*;
import com.redhat.threescale.rest.cms.ApiException;
import io.quarkus.logging.Log;
import jakarta.annotation.Nonnull;
import jakarta.annotation.Nullable;
import jakarta.inject.Inject;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.tuple.Pair;
Expand Down Expand Up @@ -108,16 +108,16 @@ public Integer call() throws Exception {
if (layoutFilename == null) {
newPageLayoutSystemName = treeDetails.getDefaultLayout()
.map(CmsLayout::getSystemName)
.orElse("");
.orElse(null);
} else if (StringUtils.isBlank(layoutFilename)) {
newPageLayoutSystemName = "";
newPageLayoutSystemName = null;
} else {
CmsObject layout = Optional.ofNullable(localObjectsByPath.get(layoutFilename))
.map(Pair::getLeft)
.orElseGet(() -> remoteObjectsByPath.get(layoutFilename));

if (layout instanceof CmsLayout) {
newPageLayoutSystemName = ((CmsLayout) layout).getSystemName();
if (layout instanceof CmsLayout cmsLayout) {
newPageLayoutSystemName = cmsLayout.getSystemName();
} else {
throw new IllegalArgumentException("Specified layout for new pages is not a layout!");
}
Expand All @@ -139,11 +139,11 @@ public Integer call() throws Exception {
uploadSortComparator = Comparator.comparing(Pair::getLeft,
sectionToTopComparator
.thenComparing((o1, o2) -> {
if (o1 instanceof CmsLayout && StringUtils.equals(newPageLayoutSystemName, ((CmsLayout) o1).getSystemName())) {
if (o1 instanceof CmsLayout layout1 && StringUtils.equals(newPageLayoutSystemName, layout1.getSystemName())) {
return -1;
}

if (o2 instanceof CmsLayout && StringUtils.equals(newPageLayoutSystemName, ((CmsLayout) o2).getSystemName())) {
if (o2 instanceof CmsLayout layout2 && StringUtils.equals(newPageLayoutSystemName, layout2.getSystemName())) {
return 1;
}

Expand All @@ -168,7 +168,7 @@ public Integer call() throws Exception {
return localObjectPair;
})
.sorted(uploadSortComparator)
.collect(Collectors.toList());
.toList();

if (noop) {
for (CmsObject object : deleteObjects) {
Expand Down Expand Up @@ -203,9 +203,9 @@ public Integer call() throws Exception {
for (Pair<CmsObject, File> pair : localObjectsToUpload) {
CmsObject object = pair.getLeft();

if (object instanceof CmsTemplate) {
if (object instanceof CmsTemplate cmsTemplate) {
Log.info("Publishing " + object.getType() + " " + pathKeyGenerator.generatePathKeyForObject(object) + "...");
client.publish((CmsTemplate) object);
client.publish(cmsTemplate);
}
}
}
Expand All @@ -218,35 +218,33 @@ private void performUpload(@Nonnull ThreescaleCmsClient client,
@Nonnull CmsObject object,
@Nonnull File file,
@Nonnull Map<String, Pair<CmsObject, File>> localFilesByPathKey,
@Nonnull Map<String, CmsObject> remoteObjectsByPathKey) throws ApiException {
@Nonnull Map<String, CmsObject> remoteObjectsByPathKey) {
String pathKey = pathKeyGenerator.generatePathKeyForObject(object);
Log.info("Uploading " + object.getType() + " " + pathKey + "...");

if (object instanceof CmsSection) {
CmsSection section = (CmsSection) object;
if (object instanceof CmsSection section) {
if (section.getParentId() == null && !"/".equals(pathKey)) {
section.setParentId(findParentId(pathKey, localFilesByPathKey, remoteObjectsByPathKey));
}
client.save(section);
} else if (object instanceof CmsFile) {
CmsFile cmsFile = (CmsFile) object;
} else if (object instanceof CmsFile cmsFile) {
if (cmsFile.getSectionId() == null) {
cmsFile.setSectionId(findParentId(pathKey, localFilesByPathKey, remoteObjectsByPathKey));
}
client.save(cmsFile, file);
} else if (object instanceof CmsTemplate) {
if (object instanceof CmsPage && ((CmsPage) object).getSectionId() == null) {
((CmsPage) object).setSectionId(findParentId(pathKey, localFilesByPathKey, remoteObjectsByPathKey));
} else if (object instanceof CmsTemplate template) {
if (template instanceof CmsPage page && page.getSectionId() == null) {
page.setSectionId(findParentId(pathKey, localFilesByPathKey, remoteObjectsByPathKey));
}
client.save((CmsTemplate) object, file);
client.save(template, file);
} else {
throw new UnsupportedOperationException("Unknown object type " + object.getClass());
}
}

private Long findParentId(@Nonnull String pathKey,
@Nonnull Map<String, Pair<CmsObject, File>> localFilesByPathKey,
@Nonnull Map<String, CmsObject> remoteObjectsByPathKey) {
@Nonnull Map<String, Pair<CmsObject, File>> localFilesByPathKey,
@Nonnull Map<String, CmsObject> remoteObjectsByPathKey) {
String parentPathKey;
if (pathKey.endsWith("/")) {
parentPathKey = pathKey.substring(0, pathKey.lastIndexOf('/', pathKey.length() - 2) + 1);
Expand All @@ -268,21 +266,18 @@ private Long findParentId(@Nonnull String pathKey,
}

parentPathKey = parentPathKey.substring(0, pathKey.lastIndexOf('/', parentPathKey.length() - 2) + 1);
} while (!"".equals(parentPathKey));
} while (!parentPathKey.isEmpty());

throw new IllegalStateException("Couldn't find any parent section ID... not even root");
}

private void setRequiredCreationProperties(@Nonnull CmsObject localObject,
@Nonnull String newPageLayoutSystemName) {
if (localObject instanceof CmsPage) {
CmsPage localPage = (CmsPage) localObject;

if ("text/html".equals(localPage.getContentType())) {
localPage.setLayout(newPageLayoutSystemName);
} else {
localPage.setLayout("");
}
@Nullable String newPageLayoutSystemName) {
if (newPageLayoutSystemName != null
&& localObject instanceof CmsPage localPage
&& "text/html".equals(localPage.getContentType())
) {
localPage.setLayout(newPageLayoutSystemName);
}
}

Expand All @@ -297,28 +292,26 @@ private void updateObjectId(@Nonnull CmsObject target,
// Only set ID... Leave the rest for 3scale to dictate what's updatable
// and not on each type of object.

if (target instanceof CmsSection) {
CmsSection targetSection = (CmsSection) target;
if (target instanceof CmsSection targetSection) {
CmsSection sourceSection = (CmsSection) source;

targetSection.setId(source.getId());
targetSection.setParentId(sourceSection.getParentId());
} else if (target instanceof CmsFile) {
CmsFile targetFile = (CmsFile) target;
} else if (target instanceof CmsFile targetFile) {
CmsFile sourceFile = (CmsFile) source;

targetFile.setId(sourceFile.getId());
targetFile.setSectionId(sourceFile.getSectionId());
} else if (target instanceof CmsLayout) {
((CmsLayout) target).setId(source.getId());
} else if (target instanceof CmsPage) {
CmsPage targetPage = (CmsPage) target;
} else if (target instanceof CmsLayout targetLayout) {
targetLayout.setId(source.getId());
} else if (target instanceof CmsPage targetPage) {
targetPage.setId(source.getId());

if (source instanceof CmsPage) {
targetPage.setSectionId(((CmsPage) source).getSectionId());
if (source instanceof CmsPage sourcePage) {
targetPage.setSectionId((sourcePage.getSectionId()));
}
} else if (target instanceof CmsPartial) {
((CmsPartial) target).setId(source.getId());
} else if (target instanceof CmsPartial targetPartial) {
targetPartial.setId(source.getId());
} else {
// Unknown... built-in pages/partials shouldn't come from local
// files anyway
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package com.fwmotion.threescale.cms.cli.config;

import com.fwmotion.threescale.cms.mixins.EnumHandlerMixIn;
import com.redhat.threescale.rest.cms.model.Error;
import com.redhat.threescale.rest.cms.model.*;
import io.quarkus.runtime.annotations.RegisterForReflection;

Expand All @@ -12,7 +13,7 @@
BuiltinPartial.class,
EnumHandler.class,
EnumTemplateType.class,
ErrorHash.class,
Error.class,
FileCreationRequest.class,
FileList.class,
FileUpdatableFields.class,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ public FileVisitResult preVisitDirectory(@Nonnull Path dir,
// If directory is not ignored, try loading .cmsignore
Path cmsIgnorePath = dir.resolve(CMSIGNORE_FILENAME);
if (cmsIgnorePath.toFile().exists()
&& cmsIgnorePath.toFile().canRead()) {
&& cmsIgnorePath.toFile().canRead()
) {
readCmsIgnoreFile(cmsIgnorePath);
} else {
ignoreRules.addLast(Collections.emptyList());
Expand Down Expand Up @@ -136,11 +137,10 @@ private boolean testPathIsIgnored(@Nonnull Path path) {
Path normalizedPath = path.normalize();

List<Boolean> matchingRules = ignoreRules.stream()
.sequential()
.flatMap(Collection::stream)
.filter(matcherEntry -> matcherEntry.getLeft().matches(normalizedPath))
.map(Pair::getRight)
.collect(Collectors.toList());
.toList();

if (matchingRules.isEmpty()) {
// Nothing matched, so not ignored
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ private static void addChildObjectsToList(LinkedList<Pair<String, CmsObject>> re
Long childParentId = GET_PARENT_ID_FUNCTIONS.getOrDefault(childObject.getClass(), o -> Long.MIN_VALUE)
.apply(childObject);

return parentId.equals(childParentId);
return Objects.equals(parentId, childParentId);
})
.peek(e -> treeWalker.add(Pair.of(e)))
.count());
Expand All @@ -72,24 +72,19 @@ public Set<String> calculateSpecifiedPaths(@Nonnull Collection<String> specified

validatePaths(specifiedPaths, objectsByPath);

switch (recurseBy) {
case NONE:
return new HashSet<>(specifiedPaths);

case PATH_PREFIX:
return specifiedPaths.stream()
return switch (recurseBy) {
case NONE -> new HashSet<>(specifiedPaths);
case PATH_PREFIX -> specifiedPaths.stream()
.flatMap(pathKey -> {
if (objectsByPath.get(pathKey).getType() == ThreescaleObjectType.SECTION) {
return objectsByPath.keySet().stream()
.filter(subKey -> StringUtils.startsWith(subKey, pathKey));
.filter(subKey -> StringUtils.startsWith(subKey, pathKey));
}

return Stream.of(pathKey);
})
.collect(Collectors.toSet());

case PARENT_ID:
return specifiedPaths.stream()
case PARENT_ID -> specifiedPaths.stream()
.flatMap(pathKey -> {
CmsObject parentObject = objectsByPath.get(pathKey);

Expand All @@ -99,13 +94,10 @@ public Set<String> calculateSpecifiedPaths(@Nonnull Collection<String> specified
addChildObjectsToList(recursingList, objectsByPath);

return recursingList.stream()
.map(Pair::getKey);
.map(Pair::getKey);
})
.collect(Collectors.toSet());

default:
throw new UnsupportedOperationException("Unknown recursion style: " + recurseBy);
}
};
}

public enum RecursionOption {
Expand Down
2 changes: 1 addition & 1 deletion docs/cli-usage.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ To use the `3scale-cms` command you need to provide a few parameters:

- An **ACCESS_TOKEN**, which can be used instead of a PROVIDER_KEY. The access
token must be granted permissions to both the Account Management API and the
Content Management API.
Developer Portal API.
- The **PROVIDER_KEY**, which can be found in the Account tab of your admin
portal (only visible to the users with "admin" role). The PROVIDER_KEY will be
ignored if an ACCESS_TOKEN is specified.
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
:sectnums:
:toc:

== 3scale Content Management API Concepts
== 3scale Developer Portal API Concepts

The 3scale Developer Portal consists of 3 primary types of objects:

Expand Down
2 changes: 1 addition & 1 deletion rest-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@
or docs
-->
<generateModels>true</generateModels>
<generateModelTests>false</generateModelTests>
<generateModelTests>true</generateModelTests>
<generateModelDocumentation>false</generateModelDocumentation>

<generateSupportingFiles>true</generateSupportingFiles>
Expand Down
2 changes: 1 addition & 1 deletion rest-client/src/main/dokka/module.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Module 3scale-cms-rest-client

A Java library for interacting with the 3scale Content Management API
A Java library for interacting with the 3scale Developer Portal API
Loading

0 comments on commit 8932df1

Please sign in to comment.