Troubleshooting Guide
This guide provides resolutions to frequently encountered issues when using Paladin Cloud. For community support, post in the #installation channel on Paladin Cloud’s Slack instance.
Installation specific questions
Error: Invalid VPC ID
An error occured(InvalidVpcID.NotFound error) when calling the DescribeVpcs operation: The vpc ID ‘vpc-1’ does not exist
Pleasre update with correct VPC/CIDR/Subnet details in common.py OR create local.py and add VPC details
Solution: You get this error when the VPC configuration is incorrect. Please check for and update the missing semi-colon in local.py
Error: Unable to locate credentials
botocore.exceptions.NoCredentialsError: Unable to locate credentials
Solution:
- Check if the IAM role is attached to the EC2 instance
- Check the error.log file
Error: Resource existence check failed.
Resource existence check failed
Please delete all the existing resources or change their names
Solution: This means there is already an installation of Paladin Cloud. Delete all existing resources to replace the existing instance or go to local.py and change the value of RESOURCE_NAME_PREFIX to create a second instance.
Error: Target Group Not Found
error updating LB Target Group…
tags: error tagging resource…
TargetGroupNotFound: Target groups…
Solution: Please run the sudo redeploy command again.
Error: “Please wait while we fetch your access roles…”
Solution 1: If deploying as HTTP, ensure the SSL certificate ARN in local.py is updated for internal deployment with HTTP and a self-signed certificate. Click here for more information on deploying Paladin Cloud with HTTP and Self Signed Certificate.
Solution 2: Verify that all tasks in ECS are running.
Solution 3: If the AWS service quota limit for AWS Fargate VCPU is less than 18, the task will not be in a running state. Check the service quota limit and request an increase.
Error: Not seeing violations immediately after installation
Solution:
Violations typically take about 30 minutes to appear. If you do not see the violations even after 30 mins, follow the steps given below:
For AWS:
Check the collector logs for the error “error getting credentials for account” by following these steps:
- Go to AWS Console -> Batch -> Job -> Search.
- Select Paladin Cloud Data from the dropdown and click AWS-Data-Collector-job.
- Click the link under the Log stream name to view log events.
- Search for Error getting Credentials for the account.
If the error is found, verify the IAM role in the target account and the IAM policy in the base account.For Azure: - Navigate to batch job, select the Paladin Cloud data queue, and search for pacbot-azure-discovery-job
- Open the job details and click log stream. This will open the CloudWatch logs for the Azure data collection job
- Search for com.microsoft.aad.adal4j.AcquireTokenCallable failed or com.microsoft.aad.adal4j.AuthenticationException
If the account setup is wrong, you’ll find the following in the logs:
2023-05-12 07:25:08 [pool-1-thread-1] ERROR c.m.aad.adal4j.AuthenticationContext - [Correlation ID: 4da4f6c0-fb54-44fc-bb70-8e03d14a18f8] Execution of class com.microsoft.aad.adal4j.AcquireTokenCallable failed.
com.microsoft.aad.adal4j.AuthenticationException: {
"error": "invalid_client",
"error_uri": "https://login.microsoftonline.com/error?code=7000215"
}
- In this case, refer to the plugin configuration screen to correctly setup the Azure account, and make sure the client secret has not expired
a. Go to Azure active directory
b. Navigate to App registration and register a new application
c. Once registered, click on Client credentials and create a new client secret
d. Get the client secret value, clientId and tenantId
For GCP
- Navigate to batch job, select the paladin-data queue, and search for pacbot-gcp-discovery-job
- Open the job details and click the log stream. This opens the CloudWatch logs for the GCP data collection job
- Search for GCPCredentialsProvider – Error or java.io.IOException: Invalid PKCS#8 data
If the account setup is wrong, you’ll find the following in the logs
2023-05-12 07:31:53 [main] ERROR c.t.p.g.i.a.GCPCredentialsProvider - Error:: {}
java.io.IOException: Invalid PKCS#8 data.
- In this case, refer to the plugin configuration screen to correctly setup the GCP account, and make sure service account keys have not expired
a. Go to GCP IAM, navigate to the service account
b. Grant this service account owner access to the project
c. Create a key for the service account in JSON format
d. Copy the contents of the JSON key file
Error: Error – Unable to create a load balancer with this ARN
Solution: Ensure that the ARN is not in a pending state for the ACM certificate. The installation will fail if the certificate is not valid.
Error: Maven build failure error
Solution: If you previously destroyed/uninstalled Paladin Cloud and encounter Maven errors during reinstallation:
If you see BUILD FAILURE in the Maven build.log file, follow these steps:
- Go to webapp/src/config/configuration.ts and replace configuration.ts with the latest release from here.
After making the changes, return to the installer directory and redeploy Paladin Cloud.
Refer to the Deploy Paladin Cloud documentation.
Error: 5XX error
Solution: Check the ECS Task
- Access the AWS console where you installed Paladin Cloud.
- Go to ECS (Elastic Container Service).
- Click on the PaladinCloud cluster.
You will see a list of 10 services. Make sure all 10 services are running. If any of the services are not running:
- Verify the service quota limit for Fargate On-Demand vCPU resource count. It should be set to a value greater than 10. If it’s less than 10, please raise a service quota increase ticket.
- Double-check the VPC you updated in local.py. If there are two CIDRs (IP address ranges), proceed to the Security Group section. Search for the PaladinCloud security group and update the Inbound Rules for HTTP and MySQL with the second CIDR.
Error: Jobs are stuck in a runnable mode in batch console, or if you notice a significant number of EC2 instances in your console
Solution: Please ensure that the AutoAssign IVP4 option is enabled in the subnet settings.
Along with the above errors, please verify the following:
- Verify that the permissions required by the installer are correct.
- Verify that the Dependencies are installed properly.
- Verify that EC2 Instance VPC and VPC for Paladin Cloud installation are the same.
- Please look at the common causes of the failures in the FAQs document here.
- If the issues persist, contact [email protected] or raise an issue here.