YAML Error: could not determine a constructor for the tag

Question

This is very similar to questions/44786412 but mine appears to be triggered by YAML safe_load(). I'm using Ruamel's library and YamlReader to glue a bunch of CloudFormation pieces together into a single, merged template. Is bang-notation just not proper YAML?

Outputs:
  Vpc:
    Value: !Ref vpc
    Export:
      Name: !Sub "${AWS::StackName}-Vpc"

No problem with these

Outputs:
  Vpc:
    Value:
      Ref: vpc
    Export:
      Name:
        Fn::Sub: "${AWS::StackName}-Vpc"

Resources:
  vpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock:
        Fn::FindInMap: [ CidrBlock, !Ref "AWS::Region", Vpc ]

---

Part 2; how to get load() to leave what's right of the 'Fn::Select:' alone.

  FromPort: 
    Fn::Select: [ 0, Fn::FindInMap: [ Service, https, Ports ] ]

gets converted to this, that now CF doesn't like.

  FromPort:
    Fn::Select: [0, {Fn::FindInMap: [Service, https, Ports]}]

If I unroll the statement fully then no problems. I guess the shorthand is just problematic.

  FromPort:
    Fn::Select:
    - 0
    - Fn::FindInMap: [Service, ssh, Ports]

Eudy

You should roll back and remove that 'Part 2'. Tagging on additional questions, even related, is not how Stack Overflow works. You could pose it as a new question, but is very unclear as it is now: which of the seven colons are you referring to? Also "gets converted to below that" , Below what? Below the colon? – Device

Answer 1

Your "bang notation" is proper YAML, normally this is called a tag. If you want to use the safe_load() with those you'll have to provide constructors for the !Ref and !Sub tags, e.g. using:

ruamel.yaml.add_constructor(u'!Ref', your_ref_constructor, constructor=ruamel.yaml.SafeConstructor)

where for both tags you should expect to handle scalars a value. and not the more common mapping.

I recommend you use the RoundTripLoader instead of the SafeLoader, that will preserve order, comments, etc. as well. The RoundTripLoader is a subclass of the SafeLoader.

If you are using ruamel.yaml>=0.15.33, which supports round-tripping scalars, you can do (using the new ruamel.yaml API):

import sys
from ruamel.yaml import YAML

yaml = YAML()
yaml.preserve_quotes = True

data = yaml.load("""\
Outputs:
  Vpc:
    Value: !Ref: vpc    # first tag
    Export:
      Name: !Sub "${AWS::StackName}-Vpc"  # second tag
""")

yaml.dump(data, sys.stdout)

to get:

Outputs:
  Vpc:
    Value: !Ref: vpc    # first tag
    Export:
      Name: !Sub "${AWS::StackName}-Vpc"  # second tag

In older 0.15.X versions, you'll have to specify the classes for the scalar objects yourself. This is cumbersome, if you have many objects, but allows for additional functionality:

import sys
from ruamel.yaml import YAML


class Ref:
    yaml_tag = u'!Ref:'

    def __init__(self, value, style=None):
        self.value = value
        self.style = style

    @classmethod
    def to_yaml(cls, representer, node):
        return representer.represent_scalar(cls.yaml_tag,
                                            u'{.value}'.format(node), node.style)

    @classmethod
    def from_yaml(cls, constructor, node):
        return cls(node.value, node.style)

    def __iadd__(self, v):
        self.value += str(v)
        return self

class Sub:
    yaml_tag = u'!Sub'
    def __init__(self, value, style=None):
        self.value = value
        self.style = style

    @classmethod
    def to_yaml(cls, representer, node):
        return representer.represent_scalar(cls.yaml_tag,
                                            u'{.value}'.format(node), node.style)

    @classmethod
    def from_yaml(cls, constructor, node):
        return cls(node.value, node.style)


yaml = YAML(typ='rt')
yaml.register_class(Ref)
yaml.register_class(Sub)

data = yaml.load("""\
Outputs:
  Vpc:
    Value: !Ref: vpc    # first tag
    Export:
      Name: !Sub "${AWS::StackName}-Vpc"  # second tag
""")

data['Outputs']['Vpc']['Value'] += '123'

yaml.dump(data, sys.stdout)

which gives:

Outputs:
  Vpc:
    Value: !Ref: vpc123 # first tag
    Export:
      Name: !Sub "${AWS::StackName}-Vpc"  # second tag

Device

Thanks. I've gotten much farther now that I've removed bang-notation and switched to using RoundTrip. But besides defining said classes (not particularly interested in defining all of CloudFormation's intrinsic functions), is there a way for Yaml to leave anything to the right of the colon alone and not parse it? – Eudy

@MatthewPatton No, the parsing needs to be done, to know where the tag ends, what the value associated with it is etc. I updated the catch-all for undefined tagged objects to support scalars (it only supported mapping based object) as you are using and updated the answer accordingly. – Device

"The RoundTripLoader is a subclass of the SafeLoader" -- is it? I can't tell from the source code: sourceforge.net/p/ruamel-yaml/code/ci/default/tree/… – Detrition

@JasonS You should probably post that as a question. – Device