Iterating over Circuit Data and Searching for certain objects¶

Here are some additional features that might be of interest, in addition to the methods shown in the previous notebook!

Selecting certain instances¶

• As shown previously, instances of a module can be selected via the Module.instances dictionary.
• Alternatively, in Module.instance_types, all instances are sorted by their types.
• Furthermore, via Module.get_instances, certain sets of instances can be selected based on string sections representing their name or type (e.g. name="inst" will return all instances whose name contains "inst").
• Execute the cell below to see different approaches for selecting sets of instances.

In [ ]:

import netlist_carpentry

circuit = netlist_carpentry.read("files/simpleAdder.v", top="simpleAdder")
top_module = circuit.top
print(f"Module {top_module} contains the following instances:")
for instance_name in top_module.instances:
    print(f"\tInstance with name {instance_name}")

print(f"Module {top_module} contains instances of the following types:")
for instance_type, instances in top_module.instances_by_types.items():
    print(f"\t{instance_type} ({len(instances)} instances)")

print("These instances have an 'e' in their name:")
instances_with_an_e_in_their_name = top_module.get_instances(name="e", fuzzy=True)
print("\t"+", ".join(inst.name for inst in instances_with_an_e_in_their_name))

print("These instances have a 'd' in their instance type descriptor:")
instances_with_a_d_in_their_type = top_module.get_instances(type="d", fuzzy=True)
print("\t"+", ".join(f"{inst.name} ({inst.instance_type})" for inst in instances_with_a_d_in_their_type))

import netlist_carpentry circuit = netlist_carpentry.read("files/simpleAdder.v", top="simpleAdder") top_module = circuit.top print(f"Module {top_module} contains the following instances:") for instance_name in top_module.instances: print(f"\tInstance with name {instance_name}") print(f"Module {top_module} contains instances of the following types:") for instance_type, instances in top_module.instances_by_types.items(): print(f"\t{instance_type} ({len(instances)} instances)") print("These instances have an 'e' in their name:") instances_with_an_e_in_their_name = top_module.get_instances(name="e", fuzzy=True) print("\t"+", ".join(inst.name for inst in instances_with_an_e_in_their_name)) print("These instances have a 'd' in their instance type descriptor:") instances_with_a_d_in_their_type = top_module.get_instances(type="d", fuzzy=True) print("\t"+", ".join(f"{inst.name} ({inst.instance_type})" for inst in instances_with_a_d_in_their_type))

Selecting certain Module Ports¶

• Similar to instances, module ports can be selected based on their name or direction.
• Execute the cell below to retrieve all ports with an "i" in their name as well as all output ports.

In [ ]:

from netlist_carpentry import Direction

print("Ports with an 'i' in their name:")
ports_with_i = top_module.get_ports(name="i", fuzzy=True)
print("\t"+", ".join(f"{p.name}" for p in ports_with_i))

print("Output Ports:")
output_ports = top_module.get_ports(direction=Direction.OUT)
print("\t"+", ".join(f"{p.name}" for p in output_ports))

from netlist_carpentry import Direction print("Ports with an 'i' in their name:") ports_with_i = top_module.get_ports(name="i", fuzzy=True) print("\t"+", ".join(f"{p.name}" for p in ports_with_i)) print("Output Ports:") output_ports = top_module.get_ports(direction=Direction.OUT) print("\t"+", ".join(f"{p.name}" for p in output_ports))

Selecting certain Wires¶

• Analogously, wires can be selected based on their name, as shown below.

In [ ]:

print("Wires with an 't' in their name:")
wires_with_t = top_module.get_wires(name="t", fuzzy=True)
print("\t"+", ".join(f"{p.name}" for p in wires_with_t))

print("Wires with special characters in their name that got replaced with '§' upon reading the Verilog file:")
wires_with_special_characters = top_module.get_wires(name="§", fuzzy=True)
print("\t"+", ".join(f"{p.name}" for p in wires_with_special_characters))

print("Wires with an 't' in their name:") wires_with_t = top_module.get_wires(name="t", fuzzy=True) print("\t"+", ".join(f"{p.name}" for p in wires_with_t)) print("Wires with special characters in their name that got replaced with '§' upon reading the Verilog file:") wires_with_special_characters = top_module.get_wires(name="§", fuzzy=True) print("\t"+", ".join(f"{p.name}" for p in wires_with_special_characters))

Selecting instances, ports or wires based on attributes¶

• Here are some approaches to retrieve objects based on a certain filter, even without dedicated methods.
• Execute the cell below to see some ways to retrieve data from all instances of a module.

In [ ]:

instances = top_module.instances.values()
all_submodule_instances = [inst for inst in instances if inst.is_module_instance]
print(f"These instances are submodule instances: {all_submodule_instances}")
all_primitive_instances = [inst for inst in instances if inst.is_primitive]
print(f"These instances are primitive instances (i.e. gates): {all_primitive_instances}")
verilog_one_liner = [inst.verilog for inst in instances if inst.verilog.count("\n") == 0]
print(f"Verilog descriptions of instances with one-line descriptions: {verilog_one_liner}")
exactly_4_ports = [inst for inst in instances if len(inst.ports) == 4]
print(f"Instances with exactly four ports: {exactly_4_ports}")
print(f"\tThose ports are: {', '.join(p.name for p in exactly_4_ports[0].ports.values())}")

instances = top_module.instances.values() all_submodule_instances = [inst for inst in instances if inst.is_module_instance] print(f"These instances are submodule instances: {all_submodule_instances}") all_primitive_instances = [inst for inst in instances if inst.is_primitive] print(f"These instances are primitive instances (i.e. gates): {all_primitive_instances}") verilog_one_liner = [inst.verilog for inst in instances if inst.verilog.count("\n") == 0] print(f"Verilog descriptions of instances with one-line descriptions: {verilog_one_liner}") exactly_4_ports = [inst for inst in instances if len(inst.ports) == 4] print(f"Instances with exactly four ports: {exactly_4_ports}") print(f"\tThose ports are: {', '.join(p.name for p in exactly_4_ports[0].ports.values())}")

• Similarly, ports (either of a module or of an instance) can be selected based on certain attributes.
• In the cell below, some approaches are shown on how to access and filter ports based on their properties.

In [ ]:

ports = top_module.ports.values()
port_names = [p.name for p in ports]
print(f"These are the names of all ports of the module: {port_names}")
ports_1bit = [p.name for p in ports if p.width == 1]
print(f"These are the names of all 1-bit wide ports: {ports_1bit}")
port_width_mapping = {p.name: p.width for p in ports}
print(f"This is a mapping from ports to their widths: {port_width_mapping}")
signal_driving_ports = [p.name for p in ports if p.is_driver]
print(f"These are the names of all ports that drive signals: {signal_driving_ports}")
signal_load_ports = [p.name for p in ports if p.is_load]
print(f"These are the names of all signal load ports: {signal_load_ports}")
fully_connected_ports = [p.name for p in ports if p.is_connected]
print(f"These are the names of all fully connected (non-floating) ports: {fully_connected_ports}")

ports = top_module.ports.values() port_names = [p.name for p in ports] print(f"These are the names of all ports of the module: {port_names}") ports_1bit = [p.name for p in ports if p.width == 1] print(f"These are the names of all 1-bit wide ports: {ports_1bit}") port_width_mapping = {p.name: p.width for p in ports} print(f"This is a mapping from ports to their widths: {port_width_mapping}") signal_driving_ports = [p.name for p in ports if p.is_driver] print(f"These are the names of all ports that drive signals: {signal_driving_ports}") signal_load_ports = [p.name for p in ports if p.is_load] print(f"These are the names of all signal load ports: {signal_load_ports}") fully_connected_ports = [p.name for p in ports if p.is_connected] print(f"These are the names of all fully connected (non-floating) ports: {fully_connected_ports}")

• Here are some more examples, on how to select wires based on certain properties.
• Most of them are similar to the examples above, but with wires instead of ports.

In [ ]:

wires = top_module.wires.values()
wire_names = [w.name for w in wires]
print(f"Here is a list of wire names: {wire_names}")
wires_1bits = [w.name for w in wires if w.width == 1]
print(f"These wires are one bit wide: {wires_1bits}")
wire_width_mapping = {w.name: w.width for w in wires}
print(f"This is a mapping from wires to their widths: {wire_width_mapping}")
wires_with_ports = {w.name: len(w.connected_port_segments) for w in wires}
print(f"These are the names of all wires mapped to their number of port segment connections (based on their width): {wires_with_ports}")
wires_with_problems = [w for w in wires if w.has_problems()]
print(f"These wires have issues (e.g. dangling or multiple drivers): {wires_with_problems}")

wires = top_module.wires.values() wire_names = [w.name for w in wires] print(f"Here is a list of wire names: {wire_names}") wires_1bits = [w.name for w in wires if w.width == 1] print(f"These wires are one bit wide: {wires_1bits}") wire_width_mapping = {w.name: w.width for w in wires} print(f"This is a mapping from wires to their widths: {wire_width_mapping}") wires_with_ports = {w.name: len(w.connected_port_segments) for w in wires} print(f"These are the names of all wires mapped to their number of port segment connections (based on their width): {wires_with_ports}") wires_with_problems = [w for w in wires if w.has_problems()] print(f"These wires have issues (e.g. dangling or multiple drivers): {wires_with_problems}")

Applying Custom Metadata to Circuit Objects¶

• Sometimes it might be useful to add some metadata to a certain object, e.g. to mark it as dont care for a certain operation, or for traversal to indicate that this objects has been visited before.
• This can be done by applying the custom metadata in a key-value manner to the NetlistElement.metadata dictionary.
• Each object of the circuit is based on the NetlistElement class, so that they all posess an metadata variable.
• By default, this dictionary is empty.
• However, Yosys may insert its own metadata, such as a link to the HDL source of this element.
• Execute the cell below to add some user data to some elements.

Info: All objects based on the class Netlist Element have the instance dictionary variables metadata and parameters. While parameters are considered in Verilog write-out since they may have a direct impact on the circuit (e.g. in regards to module or instance parameters), metadata do not affect the circuit at all and can thus be used for custom annotation or any other user-defined data.

In [ ]:

for instance in top_module.instances.values():
    if "dff" in instance.instance_type:
        instance.metadata.add("do_not_change", True)
    else:
        instance.metadata.add("do_not_change", False)
    # Or more compressed
    instance.metadata.add("do_not_change", "dff" in instance.instance_type)

print(f"These instances may be modified: {', '.join(inst.name for inst in top_module.instances.values() if not inst.metadata.get("do_not_change"))}")
print(f"These instances may NOT be modified: {', '.join(inst.name for inst in top_module.instances.values() if inst.metadata.get("do_not_change"))}")

for instance in top_module.instances.values(): if "dff" in instance.instance_type: instance.metadata.add("do_not_change", True) else: instance.metadata.add("do_not_change", False) # Or more compressed instance.metadata.add("do_not_change", "dff" in instance.instance_type) print(f"These instances may be modified: {', '.join(inst.name for inst in top_module.instances.values() if not inst.metadata.get("do_not_change"))}") print(f"These instances may NOT be modified: {', '.join(inst.name for inst in top_module.instances.values() if inst.metadata.get("do_not_change"))}")

• User-defined data could also be extracted from other heuristics.
• Execute the cell below to add custom metadata to all ports that are presumably clock ports based on their name.

Warning: Not all objects must receive a key-value pair. Often, it is sufficient to only add a corresponding entry to objects that match a certain criteria. However, when filtering the criteria later, a KeyError will arise for objects that do not have the appropriate key. The safe way is to use the dict.get method, along with a default value, which is used whenever the key is not present in the dictionary.

In [ ]:

for port in top_module.ports.values():
    port.metadata.add_category("custom_data")
    if "clk" in port.name:
        port.metadata.custom_data["presumably_clk"] = True

# Using dict.get(key, default) instead of dict[key]
print(f"These ports may be clock ports: {', '.join(port.name for port in top_module.ports.values() if port.metadata.custom_data.get('presumably_clk', False))}")
print(f"These ports may NOT be clock ports: {', '.join(port.name for port in top_module.ports.values() if not port.metadata.custom_data.get('presumably_clk', False))}")

for port in top_module.ports.values(): port.metadata.add_category("custom_data") if "clk" in port.name: port.metadata.custom_data["presumably_clk"] = True # Using dict.get(key, default) instead of dict[key] print(f"These ports may be clock ports: {', '.join(port.name for port in top_module.ports.values() if port.metadata.custom_data.get('presumably_clk', False))}") print(f"These ports may NOT be clock ports: {', '.join(port.name for port in top_module.ports.values() if not port.metadata.custom_data.get('presumably_clk', False))}")

• The method to retrieve clock ports shown in the cell above relies on ports that somehow have clk in their name.
• Another heuristic could be that signals connected to the clock port of a flip-flop presumably somehow resemble a clock signal.
• Execute the cell below to mark all wires as "clock" if they are connected to a clock port of any flip-flop.

In [ ]:

from netlist_carpentry.utils.gate_lib import DFF

# Instead of iterating over all instances, only iterate over all flip-flops.
# For the given example, this does not change much (since the module only consists of 2 instances).
# For larger modules, the iterations (and thus the total time required) may be reduced drastically.
for dff in top_module.instances_by_types["§adff"]:
    dff: DFF # To enable type checkers and IDEs to show descriptions of the attributes.
    clk_port_wire_segment_path = dff.clk_port.segments[0].ws_path
    print(f"The port {dff.clk_port.name} of DFF {dff.name} is connected to this wire segment: {clk_port_wire_segment_path.raw}")

    # Mark wire segments that clock a flip-flop as "clocks_dff"
    wire_segment = top_module.get_from_path(clk_port_wire_segment_path)
    wire_segment.metadata.add_category("dff_stuff")
    wire_segment.metadata.dff_stuff["clocks_dff"] = True

clocking_wire_segments = []
for wire in top_module.wires.values():
    for index, segment in wire:
        if segment.metadata.get("clocks_dff", False, category="dff_stuff"):
            clocking_wire_segments.append(wire)

print("These wires clock a flip-flop:")
for wire in clocking_wire_segments:
    print(f"\t{wire.raw_path}")

from netlist_carpentry.utils.gate_lib import DFF # Instead of iterating over all instances, only iterate over all flip-flops. # For the given example, this does not change much (since the module only consists of 2 instances). # For larger modules, the iterations (and thus the total time required) may be reduced drastically. for dff in top_module.instances_by_types["§adff"]: dff: DFF # To enable type checkers and IDEs to show descriptions of the attributes. clk_port_wire_segment_path = dff.clk_port.segments[0].ws_path print(f"The port {dff.clk_port.name} of DFF {dff.name} is connected to this wire segment: {clk_port_wire_segment_path.raw}") # Mark wire segments that clock a flip-flop as "clocks_dff" wire_segment = top_module.get_from_path(clk_port_wire_segment_path) wire_segment.metadata.add_category("dff_stuff") wire_segment.metadata.dff_stuff["clocks_dff"] = True clocking_wire_segments = [] for wire in top_module.wires.values(): for index, segment in wire: if segment.metadata.get("clocks_dff", False, category="dff_stuff"): clocking_wire_segments.append(wire) print("These wires clock a flip-flop:") for wire in clocking_wire_segments: print(f"\t{wire.raw_path}")

Exporting Metadata¶

• All metadata can be exported in JSON format to a file by using Module.export_metadata or Circuit.export_metadata (the latter exports the metadata of all modules).
• Alternatively, Module.normalize_metadata can be used together with several filtering options retrieve a normalized dictionary.
• The dictionary keys are the paths to the respective objects, and the associated values are dictionaries containing the metadata of the respective object, divided into the specified metadata categories.
• Execute the cell below to simply export the metadata into a file metadata.json in the output directory.
• For convenience, the result is directly read in again and displayed below.

Info: Module.export_metadata uses Module.normalize_metadata to prepare the metadata before writing it to the given file, while Circuit.export_metadata references Module.export_metadata for each module. Accordingly, all parameters of normalize_metadata can be used in the export_metadata methods, which then will simply be passed into Module.normalize_metadata.

In [ ]:

path = "output/metadata.json"
top_module.export_metadata(path)

with open(path) as f:
    metadata = f.read()
print(metadata)

path = "output/metadata.json" top_module.export_metadata(path) with open(path) as f: metadata = f.read() print(metadata)

• To sort the metadata based on the category, use the parameter sort_by of the normalize_metadata method.
• Execute the cell below to sort all metadata of the top module based on their category - the categories are custom_data, dff_stuff, general and yosys.

In [ ]:

from pprint import pprint  # For better readability

metadata = top_module.normalize_metadata(sort_by='category')
pprint(metadata)

from pprint import pprint # For better readability metadata = top_module.normalize_metadata(sort_by='category') pprint(metadata)

• The normalize_metadata method is also able to take a filter method, which allows to filter the metadata in regards to categories and metadata values.
• By default, the filter is lambda cat, md: True, which means, it ignores all category names and all associated metadata dictionary.
• Passing a lambda function lambda cat, md: cat != "yosys" when calling the method will filter out all entries, where the category is "yosys".
• Execute the cell below to receive the normalized metadata without the metadata added by yosys to the "yosys" category.

In [ ]:

metadata = top_module.normalize_metadata(sort_by='category', filter=lambda cat, md: cat != "yosys")
pprint(metadata)

metadata = top_module.normalize_metadata(sort_by='category', filter=lambda cat, md: cat != "yosys") pprint(metadata)