Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

puppet resource <...> --to_yaml mishandles structured resource values

Description

I'm specifically experiencing this with

puppet resource ec2_instance --to_yaml

using the puppetlabs-aws module from the forge. The module itself doesn't seem to be touching any yaml related stuff. Without the yaml flag I get this data: https://gist.github.com/wkalt/a97a42b84b1afc3cd23236ea32eed3fa and using the yaml flag I get https://gist.github.com/wkalt/a97a42b84b1afc3cd23236ea32eed3fa . Note that the former has ruby maps embedded, which is the bug.

Environment

None

Acceptance Criteria

None

Activity

Josh CooperOctober 16, 2019 at 3:17 PM

Thanks . I filed a new ticket https://puppet.atlassian.net/browse/PUP-10105#icft=PUP-10105. Can you comment how you're are generating that output on the new ticket, including which version of the mysql module you're using?

Iain BuclawOctober 16, 2019 at 2:35 PM
Edited

This change caused a regression. The option --to_yaml no longer generates syntactically valid YAML.

 

Before pull request #7718:

mysql_database: PERCONA_SCHEMA: ensure : 'present' charset: 'utf8' collate: 'utf8_general_ci'

After pull request #7718:

mysql_database: PERCONA_SCHEMA: ensure: present charset: !ruby/string:Puppet::Util::Execution::ProcessOutput utf8 collate: !ruby/string:Puppet::Util::Execution::ProcessOutput utf8_general_ci

Josh CooperSeptember 14, 2019 at 2:06 AM

Stephen MarlowAugust 1, 2019 at 3:00 PM
Edited

I have also been seeing this with a client. They're using `puppet resource pe_node_group` on PE which outputs:

 

pe_node_group: PE Certificate Authority: ensure : 'present' classes : { 'puppet_enterprise::profile::certificate_authority' => { } }

The classes parameter is a Hash internally.

YAML implementations can't parse the above output. As a workaround we found that a simple s/=>/:/g fixes the problem, but obviously that's not ideal.

The --to_yaml option in Puppet::Application::Resource ends up invoking Puppet::Resource.to_hierayaml. This function outputs the YAML with the following code:

 

# In Puppet::Resource.to_hierayaml ... attributes = attr.collect { |k| v = parameters[k] " %-#{attr_max}s: %s\n" % [k, Puppet::Parameter.format_value_for_display(v)] }.join ...

The Puppet::Parameter.format_value_for_display function formats the Hash in a Puppet style, which is why we get the mixed syntax. The format_value_for_display function doesn't take any additional parameters currently that would easily allow us to do YAML-specific formatting.

 

Puppet::Parameter.format_value_for_display does the following:

# In Puppet::Parameter def self.format_value_for_display(value) Puppet::Pops::Types::StringConverter.convert(value, Puppet::Pops::Types::StringConverter::DEFAULT_PARAMETER_FORMAT) end

Based on my reading, it seems that using DEFAULT_PARAMETER_FORMAT will cause the StringConvert to default to DEFAULT_HASH_FORMAT (shown below).

# In Puppet::Pops::Types::StringConverter DEFAULT_HASH_FORMAT = Format.new('%h') DEFAULT_HASH_FORMAT.separator = ', '.freeze DEFAULT_HASH_FORMAT.separator2 = ' => '.freeze DEFAULT_HASH_FORMAT.container_string_formats = DEFAULT_CONTAINER_FORMATS DEFAULT_HASH_FORMAT.freeze

This format is appropriate for Puppet code but the ' => ' separator does not work for YAML output.

One possible fix would be to provide Puppet::Pops::Types::StringConverter.convert with a format hash that overrides the separator2 field. It would also require either altering Puppet::Parameter.format_value_for_display to accept the overrides or create a new function (e.g. format_value_for_yaml) as appropriate.

I think that would be sufficient to make this work. I don't know of any other spots where the manifest output would break YAML parsing, but I also haven't done a comprehensive check on that.

 

Craig Castle-MeadNovember 11, 2018 at 8:02 AM

I've been working through converting a brownfield AWS environment to Puppet managed and there's a significant amount of nested json coming through when using the --to_yaml (-y) flag with puppet resource. Having the output correctly formatted at all layers in yaml would be a huge help.

Fixed

Details

Assignee

Reporter

Labels

Team (migrated)

Coremunity

Method Found

Needs Assessment

Sprint

Fix versions

Priority

Created August 1, 2017 at 4:47 PM
Updated March 5, 2020 at 8:32 AM
Resolved September 17, 2019 at 11:09 PM