Telemetry - Key Concepts and FAQ

Audience
Public
Product
Portworx
Content Type
Source Type
Knowledge Base

Telemetry - Key Concepts and FAQ

KB0018149
Pure Product
Portworx
Environment
SDD (Software Design Document)

High-level architecture diagram of Telemetry

 

Question
Introduction
  • Purpose: Telemetry helps in identifying potential issues before they impact the system, ensuring smooth and efficient operations.
  • Benefits: Telemetry is recommended as it offers significant advantages such as improved cluster stability, enhanced performance, and quicker resolution of issues. It benefits both the customer and Portworx by enabling better support and maintenance.
Required Network Connectivity & Ports

Telemetry has pre-requirements to be followed in order to enable this feature. 

  1.  Necessary Ports [1]
    • 9024: Telemetry log uploader
    • 12001: Telemetry phone home
    • 12002: Additional telemetry communication
    • 17001-17020: Portworx on OpenShift 4+ (TCP)
    • 17002: Portworx on OpenShift 4+ (UDP)
    • 9029: Telemetry for Portworx version 2.13.8 and newer
    • 2379: External etcd (Key-Value Database) port
  2. Security Measures:
    • Encrypted connections using HTTPS and TLS are employed to ensure secure data transmission
  3. Destination Endpoints [1]
Communication test:
  • curl -L https://register.cloud-support.purestorage.com/auth/1.0/ping
    • Expected Response: "HEADER Product-Name is required"
  • curl -L https://rest.cloud-support.purestorage.com
    • Expected Response: "Alert certificate required"
  • curl -L https://logs-01.loggly.com/
    • Expected Response: "Invalid API key"
  • curl -L https://pxessentials.portworx.com/osb/ping
    • Expected Response: "PX-OSB is UP and running"
Types of Data Collected

Telemetry will collect the following type of data:

  • Portworx journal logs
  • Output from common Portworx CLI commands that provide details about cluster, nodes, and volumes
  • Basic information about operating system of the worker nodes
  • Stack and heap files of Portworx processes
  • Alerts generated by the cluster
  • Cores or traces from Portworx processes (if found)

All Portworx customer clusters report the following technical telemetry information on a periodic basis that Portworx uses for product and technical support. Access to Pure1 is only authorized for Pure Support and customers who use Pure1 to view their deployment information. Third-party diagnostic service is used by Support and Engineering to retrieve diagnostic and usage data for product improvements and is not identifiable per customer.

Also, for Data Privacy, Telemetry does not include sensitive customer data or personally identifiable information (PII).

Data Upload Frequency

Telemetry data is sent periodically, and the frequency can be configured based on user preferences. There are also options for on-demand telemetry collection.

Security & Compliance Considerations
  • Encryption Standards: Data transmission is encrypted to protect the integrity and confidentiality of the data.
  • Compliance: Portworx adheres to industry regulations such as GDPR, SOC 2, and ISO 27001.
  • Customer Audits: Customers have the ability to audit or review the data being sent to ensure compliance with their security policies.
Enabling & Disabling Telemetry
  • Instructions: Detailed instructions for enabling or disabling telemetry and configuring telemetry settings in Portworx are provided in the documentation. [2]
  • Impact of Disabling: Disabling telemetry may affect support and troubleshooting capabilities, as it limits the ability to proactively identify and resolve issues.
Justification for Security Teams
  • Common Concerns:
    • Data Privacy: Telemetry data does not include sensitive or personally identifiable information.
    • Potential Attack Vectors: Security measures such as encryption and compliance with industry standards mitigate potential risks.
  • Justifications:
    • Improved Stability: Telemetry helps in maintaining cluster stability by providing early warnings of potential issues.
    • Enhanced Performance: Performance metrics collected through telemetry enable optimization and efficient resource management.
Answer
FAQ (Frequently Asked Questions)
Questions Answers
What do we need to configure for the connection with Pure1, and how do we configure it? The clusters have to be able to reach those endpoints: https://logs-01.loggly.com, https://rest.cloud-support.purestorage.com and https://register.cloud-support.purestorage.com. After this, you can enable Telemetry on Storage Cluster of each cluster
What is the approach for Portworx customers to upload the diagnostics to Pure1 currently? Customers need to generate the diagnostic files manually, download them from the node(s), and upload them on the Pure1 [3].
How are the diagnostics being collected by Pure1? Is it by ‘On Demand’ or ‘On Crash’? Diagnostics are sent `On crash` of Portworx or px-storage service/process or when users run pxctl service diags -a command [3].
On demand, how often will the diagnostics bundle automatically upload to Pure1? For volumes list, Portworx Cluster info & Node list and PX Alerts, they are uploaded every 30 minutes. Diagnostics are sent as per the on-crash situation or when pxctl service diags -a is executed
Could you use a more specific URL path instead of a subdomain?

The only APIs accessible on the Destination Endpoints are reserved for telemetry by Pure Storage. There are no other uses for these subdomains.

It’s not recommended to whitelist specific URLs, as we occasionally introduce new API versions and transition usage. Whitelisting specific URLs will likely cause service interruptions in the support of Portworx.
Is it possible for any user to sign up for a free account on the destination endpoints?

No
Will there be any functions/services like SMS or emails or chat or screen sharing or online meetings or remote access or VPN or P2P (peer to peer) or storage or social networking or storage access as part of the destination endpoints?

No
Will there be any application/software downloads from the destination endpoints?

No
Will there be any authentication required for accessing the destination endpoints?

Yes
What API controls are there in place related to the destination endpoints?

Clients must authenticate with these domains. Authentication requires a valid Portworx cluster license and TLS certificates with corresponding claims.
What can a client do with these destination endpoints APIs?

Clients can upload diagnostic logs and metadata using these APIs.
Are authentication credentials shared or self-registered?

Self-registered
Is it possible to move/post/upload data out of the organization via the destination endpoints APIs?

No, APIs on these subdomains are upload-only.
Why is the PUT method required?

The PUT method is required to register a cluster (during cluster deployment) and renew Telemetry auth certificate (once a year in common case)
What will be uploaded using the PUT method?

The cluster registration PUT transmits the cluster UUID, license type, and Portworx version.

The renewal PUT transmits a CSR (certificate signing request), which Pure Storage will sign and return, which is used for subsequent telemetry calls.
Will there be any usernames, passwords, hashes, or sensitive data sent using the PUT method?

No
Will there be account re-certification performed at least annually?

The Portworx cluster re-registers when its authentication credentials are close to expiry, which is annual.
Is it possible to use telemetry without the PUT method?

Without the PUT method, the cluster will never be able to successfully register, and as a consequence, the Telemetry pods will be stuck in an unhealthy state.
What is the purpose of https://logs-01.loggly.com?

Telemetry uses Loggly to send periodically diagnostic information to Pure1 
Can any of the destination endpoints be disabled? 

No, they are not decoupled in a way that allows us to turn one off without the other.

 

Additional Information
Internal Notes