CSY/reowolf Changeset - ef3eec73f359 · Centrum Wiskunde & Informatica (CWI)

Changeset - ef3eec73f359

Parent rev.

Child rev.

[Not reviewed]

0 6 1

MH - 4 years ago 2021-03-10 15:49:03
contact@maxhenger.nl

WIP on reimplementation of type table supporting polymorphic types

7 files changed with 1482 insertions and 42 deletions:

notes_max.md

src/protocol/ast.rs

src/protocol/lexer.rs

311

src/protocol/parser/mod.rs

src/protocol/parser/symbol_table.rs

src/protocol/parser/type_table.rs

src/protocol/parser/type_table2.rs

964

0 comments (0 inline, 0 general)

notes_max.md

➞

Show inline comments

 # Noteworthy changes
 - `if`, `while`, `synchronous`, `function`, `primitive` and `composite` body statements, if not yet a block, are all converted into a block for simpler parsing/compiling.
 # Thinkings: Type System
 ## Basic Types
 Builtin primitive types:
 - Boolean: `bool`
 - Unsigned integers: `u8`, `u16`, `u32`, `u64`
 - Signed integers: `i8`, `i16`, `i32`, `i64`
 - Decimal numbers: `f32`, `f64`
 - Message-based: `in<T>`, `out<T>`, `msg`
 Builtin compound types:
 - Array: `array<T>`
 User-specified types (excluding polymorphism for now):
 - Connectors: `primitive`, `composite`
 - Functions: `func`
 - Enums: `enum`
 - Unions: `enum`
 - Structs: `struct`
 ## Polymorphic Type Definitions
 Various types may be polymorphic. We will exclude builtin polymorphs from the following considerations (i.e. `in`, `out` and `array`, and their associated functions).
 Connectors, functions, enums, unions and structs may all be polymorphic. We'll use the C++/rust/Java syntax for specifying polymorphic types:
 ```pdl
 primitive fifo<T>(in<T> i, out<T> o) {
     // impl
+}
 T pick_one_of_two<T>(T one, T two) {
     // impl
+}
 enum Option<T>{ None, Some(T) }
 enum Result<T, E>{ Ok(T), Err(E) }
 struct Pair<T> { T first, T second }
 ```
 This means that during the initial validation/linker phase all of the types may have polymorphic arguments. These polyargs have just an identifier, we can only determine the concrete type when we encounter it in another body where we either defer the type or where the user has explicitly instantiated the type.
 For now we will use the C++-like trait/interfaceless polymorphism: once we know all of the polymorphic arguments we will try to monomorphize the type and check whether or not all calls make sense. This in itself is a recursive operation: inside the polymorph we may use other polymorphic types.
 ## Polymorphic Type Usage
 Within functions and connectors we may employ polymorphic types. The specification of the polymorphic arguments is not required if they can be inferred. Polymorphic arguments may be partially specified by using somekind of throwaway `_` or `auto` type. When we are monomorphizing a type we must be able to fully determine all of the polymorphic arguments, if we can't then we throw a compiler error.
 ## Conclusions
+-
     Type definitions may contain polymorphic arguments. These are simple identifiers. The polymorphic arguments are embedded in definitions, they are erased after type inference on the type's use (e.g. function call or struct instantiation) has finished.
+-
     While instantiating polymorphic types (e.g. `T something = call_a_func(3, 5)` or the inferred variant `let something = call_a_func(3, 5)`) we actually need to resolve these types. This implies:
     - Memory declarations will have their type embedded within the memory declaration itself. If we do not know the type yet, or only partially know the type, then we need to mark the type as being inferred.
     - Expressions will have their type inferred from the constraints the expression imposes on its arguments, and the types of the arguments themselves. Within an expression tree type information may flow from the root towards the leaves or from the leaves towards the root.
     This implies that type information in the AST must be fully specified. If the type information is not yet fully known then it must be embedded in some kind of datastructure that is used for type inference.
+-
     A function definition with polyargs seems to behave a bit differently: During the initial definition parsing we may want to make sure that each type in the definition either resolves to a concrete type (optionally with polymorphic arguments) or to one of the polyargs. This would also provide information to the type inference algorithm.
 ## Conclusions - Parsing
 During parsing we want to embed all of the information the programmer has provided to the compiler in the AST. So polyargs go inside type definitions, types that are supposed to be inferred are embedded in the AST, types that depend on polyargs also go into the AST. So we may have:
 - Partially concrete types with embedded inferred types (e.g. `let[] list_of_things` is an `Array<auto>` or `Result<auto> thing = Result::Ok(5)` or `let thing = Result::Ok(5)`).
 - Direct references to polyarg types (e.g. `T list_of_things` or `Result<T> thing = ...` or `T[][] array_of_arrays_yes_sir` and `Tuple2<T, T> = construct_a_thing()`)
 - Fully inferred types (e.g. `auto thing = 5` or `auto result = call_a_thing()`)
 - Completely specified types.
 ## Conclusions - Type Inference
 During type inference and typechecking we need to determine all concrete types. So we will arrive at a fully specified type: every polymorphic type will have its polyargs specified. Likewise every inferred type will be fully specified as well.
 All of this hints at having to specify two classes of types: `ParserType` and a `Type`. The `ParserType` can be:
 - A builtin type (with embedded `ParserType`s where appropriate)
 - An inferred type
 - A polyarg type
 - A symbolic type (with embedded `ParsedType`s where appropriate)
 While the final `Type` can be:
 - A builtin type (with embedded types where appropriate)
 - A known user-defined type (with embedded types where appropriate)
 Note that we cannot typecheck polymorphic connectors and functions until we monomorphize them.
@@ \ No newline at end of file @@

src/protocol/ast.rs

➞

Show inline comments

 use std::fmt;
 use std::fmt::{Debug, Display, Formatter};
 use std::ops::{Index, IndexMut};
 use super::arena::{Arena, Id};
 // use super::containers::StringAllocator;
 // TODO: @cleanup, transform wrapping types into type aliases where possible
 use crate::protocol::inputsource::*;
 /// Helper macro that defines a type alias for a AST element ID. In this case
 /// only used to alias the `Id<T>` types.
 macro_rules! define_aliased_ast_id {
     ($name:ident, $parent:ty) => {
         pub type $name = $parent;
+    }
+}
 /// Helper macro that defines a subtype for a particular variant of an AST
 /// element ID.
 macro_rules! define_new_ast_id {
     ($name:ident, $parent:ty) => {
         #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
         pub struct $name (pub(crate) $parent);
         impl $name {
             pub fn upcast(self) -> $parent {
                 self.0
+            }
+        }
     };
+}
 define_aliased_ast_id!(RootId, Id<Root>);
 define_aliased_ast_id!(PragmaId, Id<Pragma>);
 define_aliased_ast_id!(ImportId, Id<Import>);
-define_aliased_ast_id!(TypeAnnotationId, Id<TypeAnnotation>);
+define_aliased_ast_id!(ParserTypeId, Id<ParserType>);
 define_aliased_ast_id!(VariableId, Id<Variable>);
 define_new_ast_id!(ParameterId, VariableId);
 define_new_ast_id!(LocalId, VariableId);
 define_aliased_ast_id!(DefinitionId, Id<Definition>);
 define_new_ast_id!(StructId, DefinitionId);
 define_new_ast_id!(EnumId, DefinitionId);
 define_new_ast_id!(ComponentId, DefinitionId);
 define_new_ast_id!(FunctionId, DefinitionId);
 define_aliased_ast_id!(StatementId, Id<Statement>);
 define_new_ast_id!(BlockStatementId, StatementId);
 define_new_ast_id!(LocalStatementId, StatementId);
 define_new_ast_id!(MemoryStatementId, LocalStatementId);
 define_new_ast_id!(ChannelStatementId, LocalStatementId);
 define_new_ast_id!(SkipStatementId, StatementId);
 define_new_ast_id!(LabeledStatementId, StatementId);
 define_new_ast_id!(IfStatementId, StatementId);
 define_new_ast_id!(EndIfStatementId, StatementId);
 define_new_ast_id!(WhileStatementId, StatementId);
 define_new_ast_id!(EndWhileStatementId, StatementId);
 define_new_ast_id!(BreakStatementId, StatementId);
 define_new_ast_id!(ContinueStatementId, StatementId);
 define_new_ast_id!(SynchronousStatementId, StatementId);
 define_new_ast_id!(EndSynchronousStatementId, StatementId);
 define_new_ast_id!(ReturnStatementId, StatementId);
 define_new_ast_id!(AssertStatementId, StatementId);
 define_new_ast_id!(GotoStatementId, StatementId);
 define_new_ast_id!(NewStatementId, StatementId);
 define_new_ast_id!(PutStatementId, StatementId);
 define_new_ast_id!(ExpressionStatementId, StatementId);
 define_aliased_ast_id!(ExpressionId, Id<Expression>);
 define_new_ast_id!(AssignmentExpressionId, ExpressionId);
 define_new_ast_id!(ConditionalExpressionId, ExpressionId);
 define_new_ast_id!(BinaryExpressionId, ExpressionId);
 define_new_ast_id!(UnaryExpressionId, ExpressionId);
 define_new_ast_id!(IndexingExpressionId, ExpressionId);
 define_new_ast_id!(SlicingExpressionId, ExpressionId);
 define_new_ast_id!(SelectExpressionId, ExpressionId);
 define_new_ast_id!(ArrayExpressionId, ExpressionId);
 define_new_ast_id!(ConstantExpressionId, ExpressionId);
 define_new_ast_id!(CallExpressionId, ExpressionId);
 define_new_ast_id!(VariableExpressionId, ExpressionId);
 // TODO: @cleanup - pub qualifiers can be removed once done
 #[derive(Debug, serde::Serialize, serde::Deserialize)]
 pub struct Heap {
     // Allocators
     // #[serde(skip)] string_alloc: StringAllocator,
     // Root arena, contains the entry point for different modules. Each root
     // contains lists of IDs that correspond to the other arenas.
     pub(crate) protocol_descriptions: Arena<Root>,
     // Contents of a file, these are the elements the `Root` elements refer to
     pragmas: Arena<Pragma>,
     pub(crate) imports: Arena<Import>,
     identifiers: Arena<Identifier>,
-    pub(crate) type_annotations: Arena<TypeAnnotation>,
+    pub(crate) parser_types: Arena<ParserType>,
     pub(crate) variables: Arena<Variable>,
     pub(crate) definitions: Arena<Definition>,
     pub(crate) statements: Arena<Statement>,
     pub(crate) expressions: Arena<Expression>,
+}
 impl Heap {
     pub fn new() -> Heap {
         Heap {
             // string_alloc: StringAllocator::new(),
             protocol_descriptions: Arena::new(),
             pragmas: Arena::new(),
             imports: Arena::new(),
             identifiers: Arena::new(),
-            type_annotations: Arena::new(),
+            parser_types: Arena::new(),
             variables: Arena::new(),
             definitions: Arena::new(),
             statements: Arena::new(),
             expressions: Arena::new(),
+        }
+    }
-    pub fn alloc_type_annotation(
+    pub fn alloc_parser_type(
         &mut self,
         f: impl FnOnce(TypeAnnotationId) -> TypeAnnotation,
     ) -> TypeAnnotationId {
         self.type_annotations.alloc_with_id(|id| f(id))
         f: impl FnOnce(ParserTypeId) -> ParserType,
     ) -> ParserTypeId {
         self.parser_types.alloc_with_id(|id| f(id))
+    }
     pub fn alloc_parameter(&mut self, f: impl FnOnce(ParameterId) -> Parameter) -> ParameterId {
         ParameterId(
             self.variables.alloc_with_id(|id| Variable::Parameter(f(ParameterId(id)))),
+        )
+    }
     pub fn alloc_local(&mut self, f: impl FnOnce(LocalId) -> Local) -> LocalId {
         LocalId(
             self.variables.alloc_with_id(|id| Variable::Local(f(LocalId(id)))),
+        )
+    }
     pub fn alloc_assignment_expression(
         &mut self,
         f: impl FnOnce(AssignmentExpressionId) -> AssignmentExpression,
     ) -> AssignmentExpressionId {
         AssignmentExpressionId(
             self.expressions.alloc_with_id(|id| {
                 Expression::Assignment(f(AssignmentExpressionId(id)))
             })
+        )
+    }
     pub fn alloc_conditional_expression(
         &mut self,
         f: impl FnOnce(ConditionalExpressionId) -> ConditionalExpression,
     ) -> ConditionalExpressionId {
         ConditionalExpressionId(
             self.expressions.alloc_with_id(|id| {
                 Expression::Conditional(f(ConditionalExpressionId(id)))
             })
+        )
+    }
     pub fn alloc_binary_expression(
         &mut self,
         f: impl FnOnce(BinaryExpressionId) -> BinaryExpression,
     ) -> BinaryExpressionId {
         BinaryExpressionId(
             self.expressions
                 .alloc_with_id(|id| Expression::Binary(f(BinaryExpressionId(id)))),
+        )
+    }
     pub fn alloc_unary_expression(
         &mut self,
         f: impl FnOnce(UnaryExpressionId) -> UnaryExpression,
     ) -> UnaryExpressionId {
         UnaryExpressionId(
             self.expressions
                 .alloc_with_id(|id| Expression::Unary(f(UnaryExpressionId(id)))),
+        )
+    }
@@ @@ -424,96 +423,103 @@ impl Heap { @@
             self.definitions
                 .alloc_with_id(|id| Definition::Function(f(FunctionId(id)))),
+        )
+    }
     pub fn alloc_pragma(&mut self, f: impl FnOnce(PragmaId) -> Pragma) -> PragmaId {
         self.pragmas.alloc_with_id(|id| f(id))
+    }
     pub fn alloc_import(&mut self, f: impl FnOnce(ImportId) -> Import) -> ImportId {
         self.imports.alloc_with_id(|id| f(id))
+    }
     pub fn alloc_protocol_description(&mut self, f: impl FnOnce(RootId) -> Root) -> RootId {
         self.protocol_descriptions.alloc_with_id(|id| f(id))
+    }
+}
 impl Index<RootId> for Heap {
     type Output = Root;
     fn index(&self, index: RootId) -> &Self::Output {
         &self.protocol_descriptions[index]
+    }
+}
 impl IndexMut<RootId> for Heap {
     fn index_mut(&mut self, index: RootId) -> &mut Self::Output {
         &mut self.protocol_descriptions[index]
+    }
+}
 impl Index<PragmaId> for Heap {
     type Output = Pragma;
     fn index(&self, index: PragmaId) -> &Self::Output {
         &self.pragmas[index]
+    }
+}
 impl Index<ImportId> for Heap {
     type Output = Import;
     fn index(&self, index: ImportId) -> &Self::Output {
         &self.imports[index]
+    }
+}
 impl IndexMut<ImportId> for Heap {
     fn index_mut(&mut self, index: ImportId) -> &mut Self::Output {
         &mut self.imports[index]
+    }
+}
 impl Index<ParserTypeId> for Heap {
     type Output = ParserType;
     fn index(&self, index: ParserTypeId) -> &Self::Output {
         &self.parser_types[index.index]
+    }
+}
 impl Index<TypeAnnotationId> for Heap {
     type Output = TypeAnnotation;
     fn index(&self, index: TypeAnnotationId) -> &Self::Output {
         &self.type_annotations[index]
+    }
+}
 impl Index<VariableId> for Heap {
     type Output = Variable;
     fn index(&self, index: VariableId) -> &Self::Output {
         &self.variables[index]
+    }
+}
 impl Index<ParameterId> for Heap {
     type Output = Parameter;
     fn index(&self, index: ParameterId) -> &Self::Output {
         &self.variables[index.0].as_parameter()
+    }
+}
 impl Index<LocalId> for Heap {
     type Output = Local;
     fn index(&self, index: LocalId) -> &Self::Output {
         &self.variables[index.0].as_local()
+    }
+}
 impl IndexMut<LocalId> for Heap {
     fn index_mut(&mut self, index: LocalId) -> &mut Self::Output {
         self.variables[index.0].as_local_mut()
+    }
+}
 impl Index<DefinitionId> for Heap {
     type Output = Definition;
     fn index(&self, index: DefinitionId) -> &Self::Output {
         &self.definitions[index]
+    }
+}
 impl Index<ComponentId> for Heap {
     type Output = Component;
     fn index(&self, index: ComponentId) -> &Self::Output {
         &self.definitions[index.0].as_component()
+    }
+}
@@ @@ -983,96 +989,166 @@ pub(crate) struct NamespacedIdentifierIter<'a> { @@
+}
 impl<'a> NamespacedIdentifierIter<'a> {
     pub(crate) fn num_returned(&self) -> u8 {
         return self.num_returned;
+    }
     pub(crate) fn num_remaining(&self) -> u8 {
         return self.num_total - self.num_returned
+    }
+}
 impl<'a> Iterator for NamespacedIdentifierIter<'a> {
     type Item = &'a [u8];
     fn next(&mut self) -> Option<Self::Item> {
         if self.cur_offset >= self.value.len() {
             debug_assert_eq!(self.num_returned, self.num_total);
             None
         } else {
             debug_assert!(self.num_returned < self.num_total);
             let start = self.cur_offset;
             let mut end = start;
             while end < self.value.len() - 1 {
                 if self.value[end] == b':' && self.value[end + 1] == b':' {
                     self.cur_offset = end + 2;
                     self.num_returned += 1;
                     return Some(&self.value[start..end]);
+                }
                 end += 1;
+            }
             // If NamespacedIdentifier is constructed properly, then we cannot
             // end with "::" in the value, so
             debug_assert!(end == 0 || (self.value[end - 1] != b':' && self.value[end] != b':'));
             debug_assert_eq!(self.num_returned + 1, self.num_total);
             self.cur_offset = self.value.len();
             self.num_returned += 1;
             return Some(&self.value[start..]);
+        }
+    }
+}
 impl Display for Identifier {
     fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
         // A source identifier is in ASCII range.
         write!(f, "{}", String::from_utf8_lossy(&self.value))
+    }
+}
 /// TODO: @cleanup Maybe handle this differently, preallocate in heap? The
 ///     reason I'm handling it like this now is so we don't allocate types in
 ///     the `Arena` structure if they're the common types defined here.
 pub enum ParserTypeVariant {
     // Basic builtin
     Message,
     Bool,
     Byte,
     Short,
     Int,
     Long,
     String,
     // Literals (need to get concrete builtin type during typechecking)
     IntegerLiteral,
     Inferred,
     // Complex builtins
     Array(ParserTypeId), // array of a type
     Input(ParserTypeId), // typed input endpoint of a channel
     Output(ParserTypeId), // typed output endpoint of a channel
     Symbolic(SymbolicParserType), // symbolic type (definition or polyarg)
+}
 impl ParserTypeVariant {
     pub(crate) fn supports_polymorphic_args(&self) -> bool {
         use ParserTypeVariant::*;
         match ParserTypeVariant {
             Message | Bool | Byte | Short | Int | Long | String | IntegerLiteral | Inferred => false,
             _ => true
+        }
+    }
+}
 /// ParserType is a specification of a type during the parsing phase and initial
 /// linker/validator phase of the compilation process. These types may be
 /// (partially) inferred or represent literals (e.g. a integer whose bytesize is
 /// not yet determined).
 #[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
 pub struct ParserType {
     pub this: ParserTypeId,
     pub pos: InputPosition,
     pub variant: ParserTypeVariant,
+}
 /// SymbolicParserType is the specification of a symbolic type. During the
 /// parsing phase we will only store the identifier of the type. During the
 /// validation phase we will determine whether it refers to a user-defined type,
 /// or a polymorphic argument. After the validation phase it may still be the
 /// case that the resulting `variant` will not pass the typechecker.
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct SymbolicParserType {
     // Phase 1: parser
     pub identifier: NamespacedIdentifier,
     /// The user-specified polymorphic arguments. Zero-length implies that the
     /// user did not specify any of them, and they're either not needed or all
     /// need to be inferred. Otherwise the number of polymorphic arguments must
     /// match those of the corresponding definition
     pub poly_args: Vec<ParserTypeId>,
     // Phase 2: validation/linking
     pub variant: Option<SymbolicParserTypeVariant>
+}
 /// Specifies whether the symbolic type points to an actual user-defined type,
 /// or whether it points to a polymorphic argument within the definition (e.g.
 /// a defined variable `T var` within a function `int func<T>()`
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum SymbolicParserTypeVariant {
     Definition(DefinitionId),
     PolyArg((DefinitionId, u32)), // index of polyarg in the definition
+}
 #[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
 pub enum PrimitiveType {
     Input,
     Output,
     Message,
     Boolean,
     Byte,
     Short,
     Int,
     Long,
     Symbolic(PrimitiveSymbolic)
+}
 // TODO: @cleanup, remove PartialEq implementations
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct PrimitiveSymbolic {
     // Phase 1: parser
     pub(crate) identifier: NamespacedIdentifier,
     // Phase 2: typing
     pub(crate) definition: Option<DefinitionId>
+}
 impl PartialEq for PrimitiveSymbolic {
     fn eq(&self, other: &Self) -> bool {
         self.identifier == other.identifier
+    }
+}
 impl Eq for PrimitiveSymbolic{}
 #[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
 pub struct Type {
     pub primitive: PrimitiveType,
     pub array: bool,
+}
 #[allow(dead_code)]
 impl Type {
     pub const INPUT: Type = Type { primitive: PrimitiveType::Input, array: false };
     pub const OUTPUT: Type = Type { primitive: PrimitiveType::Output, array: false };
     pub const MESSAGE: Type = Type { primitive: PrimitiveType::Message, array: false };
     pub const BOOLEAN: Type = Type { primitive: PrimitiveType::Boolean, array: false };
     pub const BYTE: Type = Type { primitive: PrimitiveType::Byte, array: false };
     pub const SHORT: Type = Type { primitive: PrimitiveType::Short, array: false };
     pub const INT: Type = Type { primitive: PrimitiveType::Int, array: false };
     pub const LONG: Type = Type { primitive: PrimitiveType::Long, array: false };
     pub const INPUT_ARRAY: Type = Type { primitive: PrimitiveType::Input, array: true };
     pub const OUTPUT_ARRAY: Type = Type { primitive: PrimitiveType::Output, array: true };
@@ @@ -1249,112 +1325,112 @@ impl Variable { @@
+        }
+    }
     pub fn is_parameter(&self) -> bool {
         match self {
             Variable::Parameter(_) => true,
             _ => false,
+        }
+    }
     pub fn as_parameter(&self) -> &Parameter {
         match self {
             Variable::Parameter(result) => result,
             _ => panic!("Unable to cast `Variable` to `Parameter`"),
+        }
+    }
     pub fn as_local(&self) -> &Local {
         match self {
             Variable::Local(result) => result,
             _ => panic!("Unable to cast `Variable` to `Local`"),
+        }
+    }
     pub fn as_local_mut(&mut self) -> &mut Local {
         match self {
             Variable::Local(result) => result,
             _ => panic!("Unable to cast 'Variable' to 'Local'"),
+        }
+    }
     pub fn the_type<'b>(&self, h: &'b Heap) -> &'b Type {
         match self {
             Variable::Parameter(param) => &h[param.type_annotation].the_type,
             Variable::Local(local) => &h[local.type_annotation].the_type,
+        }
+    }
+}
 impl SyntaxElement for Variable {
     fn position(&self) -> InputPosition {
         match self {
             Variable::Parameter(decl) => decl.position(),
             Variable::Local(decl) => decl.position(),
+        }
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Parameter {
     pub this: ParameterId,
     // Phase 1: parser
     pub position: InputPosition,
-    pub type_annotation: TypeAnnotationId,
+    pub parser_type: ParserTypeId,
     pub identifier: Identifier,
+}
 impl SyntaxElement for Parameter {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Local {
     pub this: LocalId,
     // Phase 1: parser
     pub position: InputPosition,
-    pub type_annotation: TypeAnnotationId,
+    pub parser_type: ParserTypeId,
     pub identifier: Identifier,
     // Phase 2: linker
     pub relative_pos_in_block: u32,
+}
 impl SyntaxElement for Local {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Definition {
     Struct(StructDefinition),
     Enum(EnumDefinition),
     Component(Component),
     Function(Function),
+}
 impl Definition {
     pub fn is_struct(&self) -> bool {
         match self {
             Definition::Struct(_) => true,
             _ => false
+        }
+    }
     pub fn as_struct(&self) -> &StructDefinition {
         match self {
             Definition::Struct(result) => result,
             _ => panic!("Unable to cast 'Definition' to 'StructDefinition'"),
+        }
+    }
     pub fn is_enum(&self) -> bool {
         match self {
             Definition::Enum(_) => true,
             _ => false,
+        }
+    }
     pub fn as_enum(&self) -> &EnumDefinition {
         match self {
             Definition::Enum(result) => result,
             _ => panic!("Unable to cast 'Definition' to 'EnumDefinition'"),
+        }
+    }
     pub fn is_component(&self) -> bool {
         match self {
             Definition::Component(_) => true,
             _ => false,
+        }
@@ @@ -1383,161 +1459,165 @@ impl Definition { @@
         // TODO: Fix this
         static EMPTY_VEC: Vec<ParameterId> = Vec::new();
         match self {
             Definition::Component(com) => &com.parameters,
             Definition::Function(fun) => &fun.parameters,
             _ => &EMPTY_VEC,
+        }
+    }
     pub fn body(&self) -> StatementId {
         // TODO: Fix this
         match self {
             Definition::Component(com) => com.body,
             Definition::Function(fun) => fun.body,
             _ => panic!("cannot retrieve body (for EnumDefinition or StructDefinition)")
+        }
+    }
+}
 impl SyntaxElement for Definition {
     fn position(&self) -> InputPosition {
         match self {
             Definition::Struct(def) => def.position,
             Definition::Enum(def) => def.position,
             Definition::Component(def) => def.position(),
             Definition::Function(def) => def.position(),
+        }
+    }
+}
 impl VariableScope for Definition {
     fn parent_scope(&self, _h: &Heap) -> Option<Scope> {
         None
+    }
     fn get_variable(&self, h: &Heap, id: &Identifier) -> Option<VariableId> {
         for &parameter_id in self.parameters().iter() {
             let parameter = &h[parameter_id];
             if parameter.identifier.value == id.value {
                 return Some(parameter_id.0);
+            }
+        }
         None
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct StructFieldDefinition {
     pub position: InputPosition,
     pub field: Identifier,
-    pub the_type: TypeAnnotationId,
+    pub parser_type: ParserTypeId,
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct StructDefinition {
     pub this: StructId,
     // Phase 1: parser
     pub position: InputPosition,
     pub identifier: Identifier,
     pub poly_vars: Vec<Identifier>,
     pub fields: Vec<StructFieldDefinition>
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize, PartialEq)]
 pub enum EnumVariantValue {
     None,
     Integer(i64),
-    Type(TypeAnnotationId),
+    Type(ParserTypeId),
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct EnumVariantDefinition {
     pub position: InputPosition,
     pub identifier: Identifier,
     pub value: EnumVariantValue,
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct EnumDefinition {
     pub this: EnumId,
     // Phase 1: parser
     pub position: InputPosition,
     pub identifier: Identifier,
     pub poly_vars: Vec<Identifier>,
     pub variants: Vec<EnumVariantDefinition>,
+}
 #[derive(Debug, Clone, Copy, serde::Serialize, serde::Deserialize)]
 pub enum ComponentVariant {
     Primitive,
     Composite,
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Component {
     pub this: ComponentId,
     // Phase 1: parser
     pub position: InputPosition,
     pub variant: ComponentVariant,
     pub identifier: Identifier,
     pub poly_vars: Vec<Identifier>,
     pub parameters: Vec<ParameterId>,
     pub body: StatementId,
+}
 impl SyntaxElement for Component {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Function {
     pub this: FunctionId,
     // Phase 1: parser
     pub position: InputPosition,
-    pub return_type: TypeAnnotationId,
+    pub return_type: ParserTypeId,
     pub identifier: Identifier,
     pub poly_vars: Vec<Identifier>,
     pub parameters: Vec<ParameterId>,
     pub body: StatementId,
+}
 impl SyntaxElement for Function {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Signature {
     Component(ComponentSignature),
     Function(FunctionSignature),
+}
 impl Signature {
     pub fn from_definition(h: &Heap, def: DefinitionId) -> Signature {
         // TODO: Fix this
         match &h[def] {
             Definition::Component(com) => Signature::Component(ComponentSignature {
                 identifier: com.identifier.clone(), // TODO: @fix
                 arity: Signature::convert_parameters(h, &com.parameters),
             }),
             Definition::Function(fun) => Signature::Function(FunctionSignature {
                 return_type: h[fun.return_type].the_type.clone(),
                 identifier: fun.identifier.clone(), // TODO: @fix
                 arity: Signature::convert_parameters(h, &fun.parameters),
             }),
             _ => panic!("cannot retrieve signature (for StructDefinition or EnumDefinition)")
+        }
+    }
     fn convert_parameters(h: &Heap, params: &Vec<ParameterId>) -> Vec<Type> {
         let mut result = Vec::new();
         for &param in params.iter() {
             result.push(h[h[param].type_annotation].the_type.clone());
+        }
         result
+    }
     fn identifier(&self) -> &Identifier {
         match self {
             Signature::Component(com) => &com.identifier,
             Signature::Function(fun) => &fun.identifier,
+        }
+    }
     pub fn is_component(&self) -> bool {
         match self {
             Signature::Component(_) => true,
@@ @@ -1885,96 +1965,101 @@ impl LocalStatement { @@
+        }
+    }
     pub fn as_memory(&self) -> &MemoryStatement {
         match self {
             LocalStatement::Memory(result) => result,
             _ => panic!("Unable to cast `LocalStatement` to `MemoryStatement`"),
+        }
+    }
     pub fn as_channel(&self) -> &ChannelStatement {
         match self {
             LocalStatement::Channel(result) => result,
             _ => panic!("Unable to cast `LocalStatement` to `ChannelStatement`"),
+        }
+    }
     pub fn next(&self) -> Option<StatementId> {
         match self {
             LocalStatement::Memory(stmt) => stmt.next,
             LocalStatement::Channel(stmt) => stmt.next,
+        }
+    }
+}
 impl SyntaxElement for LocalStatement {
     fn position(&self) -> InputPosition {
         match self {
             LocalStatement::Memory(stmt) => stmt.position(),
             LocalStatement::Channel(stmt) => stmt.position(),
+        }
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct MemoryStatement {
     pub this: MemoryStatementId,
     // Phase 1: parser
     pub position: InputPosition,
     pub variable: LocalId,
     pub initial: ExpressionId,
     // Phase 2: linker
     pub next: Option<StatementId>,
+}
 impl SyntaxElement for MemoryStatement {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 /// ChannelStatement is the declaration of an input and output port associated
 /// with the same channel. Note that the polarity of the ports are from the
 /// point of view of the component. So an output port is something that a
 /// component uses to send data over (i.e. it is the "input end" of the
 /// channel), and vice versa.
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct ChannelStatement {
     pub this: ChannelStatementId,
     // Phase 1: parser
     pub position: InputPosition,
     pub from: LocalId, // output
     pub to: LocalId,   // input
     // Phase 2: linker
     pub relative_pos_in_block: u32,
     pub next: Option<StatementId>,
+}
 impl SyntaxElement for ChannelStatement {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct SkipStatement {
     pub this: SkipStatementId,
     // Phase 1: parser
     pub position: InputPosition,
     // Phase 2: linker
     pub next: Option<StatementId>,
+}
 impl SyntaxElement for SkipStatement {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct LabeledStatement {
     pub this: LabeledStatementId,
     // Phase 1: parser
     pub position: InputPosition,
     pub label: Identifier,
     pub body: StatementId,
     // Phase 2: linker
     pub relative_pos_in_block: u32,
     pub in_sync: Option<SynchronousStatementId>,
+}
 impl SyntaxElement for LabeledStatement {
     fn position(&self) -> InputPosition {
         self.position

src/protocol/lexer.rs

➞

Show inline comments

@@ @@ -239,257 +239,483 @@ impl Lexer<'_> { @@
             let parsed = if data[1] == b'b' {
                 let data = String::from_utf8_lossy(&data[2..]);
                 i64::from_str_radix(&data, 2)
             } else if data[1] == b'o' {
                 let data = String::from_utf8_lossy(&data[2..]);
                 i64::from_str_radix(&data, 8)
             } else if data[1] == b'x' {
                 let data = String::from_utf8_lossy(&data[2..]);
                 i64::from_str_radix(&data, 16)
             } else {
                 // Assume decimal
                 let data = String::from_utf8_lossy(&data);
                 i64::from_str_radix(&data, 10)
             };
             if let Err(_err) = parsed {
                 return Err(ParseError2::new_error(&self.source, position, "Invalid integer constant"));
+            }
             Ok(parsed.unwrap())
+        }
+    }
     // Statement keywords
     // TODO: Clean up these functions
     fn has_statement_keyword(&self) -> bool {
         self.has_keyword(b"channel")
             || self.has_keyword(b"skip")
             || self.has_keyword(b"if")
             || self.has_keyword(b"while")
             || self.has_keyword(b"break")
             || self.has_keyword(b"continue")
             || self.has_keyword(b"synchronous")
             || self.has_keyword(b"return")
             || self.has_keyword(b"assert")
             || self.has_keyword(b"goto")
             || self.has_keyword(b"new")
             || self.has_keyword(b"put") // TODO: @fix, should be a function, even though it has sideeffects
+    }
     fn has_type_keyword(&self) -> bool {
         self.has_keyword(b"in")
             || self.has_keyword(b"out")
             || self.has_keyword(b"msg")
             || self.has_keyword(b"boolean")
             || self.has_keyword(b"byte")
             || self.has_keyword(b"short")
             || self.has_keyword(b"int")
             || self.has_keyword(b"long")
             || self.has_keyword(b"auto")
+    }
     fn has_builtin_keyword(&self) -> bool {
         self.has_keyword(b"get")
             || self.has_keyword(b"fires")
             || self.has_keyword(b"create")
             || self.has_keyword(b"length")
+    }
     fn has_reserved(&self) -> bool {
         self.has_statement_keyword()
             || self.has_type_keyword()
             || self.has_builtin_keyword()
             || self.has_keyword(b"let")
             || self.has_keyword(b"struct")
             || self.has_keyword(b"enum")
             || self.has_keyword(b"true")
             || self.has_keyword(b"false")
             || self.has_keyword(b"null")
+    }
     // Identifiers
     fn has_identifier(&self) -> bool {
         if self.has_statement_keyword() || self.has_type_keyword() || self.has_builtin_keyword() {
             return false;
+        }
         let next = self.source.next();
         is_ident_start(next)
+    }
     fn consume_identifier(&mut self) -> Result<Identifier, ParseError2> {
         if self.has_statement_keyword() || self.has_type_keyword() || self.has_builtin_keyword() {
             return Err(self.error_at_pos("Expected identifier"));
+        }
         let position = self.source.pos();
         let value = self.consume_ident()?;
         Ok(Identifier{ position, value })
+    }
     fn consume_identifier_spilled(&mut self) -> Result<(), ParseError2> {
         if self.has_statement_keyword() || self.has_type_keyword() || self.has_builtin_keyword() {
             return Err(self.error_at_pos("Expected identifier"));
+        }
         self.consume_ident()?;
         Ok(())
+    }
     fn has_namespaced_identifier(&self) -> bool {
         self.has_identifier()
+    }
     fn consume_namespaced_identifier(&mut self) -> Result<NamespacedIdentifier, ParseError2> {
         if self.has_reserved() {
             return Err(self.error_at_pos("Encountered reserved keyword"));
+        }
         let position = self.source.pos();
         let mut ns_ident = self.consume_ident()?;
         let mut num_namespaces = 1;
         while self.has_string(b"::") {
             if num_namespaces >= MAX_NAMESPACES {
                 return Err(self.error_at_pos("Too many namespaces in identifier"));
+            }
             let new_ident = self.consume_ident()?;
             ns_ident.extend(b"::");
             ns_ident.extend(new_ident);
             num_namespaces += 1;
+        }
         Ok(NamespacedIdentifier{
             position,
             value: ns_ident,
             num_namespaces,
         })
+    }
     // Types and type annotations
     /// Consumes a type definition. When called the input position should be at
     /// the type specification. When done the input position will be at the end
     /// of the type specifications (hence may be at whitespace).
     fn consume_type2(&mut self, h: &mut Heap, allow_inference: bool) -> Result<ParserTypeId, ParseError2> {
         // Small helper function to convert in/out polymorphic arguments
         let reduce_port_poly_args = |
             heap: &mut Heap,
             port_pos: &InputPosition,
             args: Vec<ParserTypeId>,
         | {
             match args.len() {
 => Ok(h.alloc_parser_type(|this| ParserType{
                         this,
                         pos: port_pos.clone(),
                         variant: ParserTypeVariant::Inferred
                 })),
 => Ok(args[0]),
                 _ => Err(())
+            }
         };
         // Consume the type
         let pos = self.source.pos();
         let parser_type_variant = if self.has_keyword(b"msg") {
             self.consume_keyword(b"msg");
             ParserTypeVariant::Message
         } else if self.has_keyword(b"boolean") {
             self.consume_keyword(b"boolean");
             ParserTypeVariant::Bool
         } else if self.has_keyword(b"byte") {
             self.consume_keyword(b"byte");
             ParserTypeVariant::Byte
         } else if self.has_keyword(b"short") {
             self.consume_keyword(b"short");
             ParserTypeVariant::Short
         } else if self.has_keyword(b"int") {
             self.consume_keyword(b"int");
             ParserTypeVariant::Int
         } else if self.has_keyword(b"long") {
             self.consume_keyword(b"long");
             ParserTypeVariant::Long
         } else if self.has_keyword(b"str") {
             self.consume_keyword(b"str");
             ParserTypeVariant::String
         } else if self.has_keyword(b"auto") {
             if !allow_inference {
                 return Err(ParseError2::new_error(
                         &self.source, pos,
                         "Type inference is not allowed here"
                 ));
+            }
             self.consume_keyword(b"auto");
             ParserTypeVariant::Inferred
         } else if self.has_keyword(b"in") {
             // TODO: @cleanup: not particularly neat to have this special case
             //  where we enforce polyargs in the parser-phase
             self.consume_keyword(b"in");
             let poly_args = self.consume_polymorphic_args(h, allow_inference)?;
             let poly_arg = reduce_port_poly_args(h, &pos, poly_args)
                 .map_err(|| ParseError2::new_error(
                     &self.source, pos, "'in' type only accepts up to 1 polymorphic argument"
                 ))?;
             ParserTypeVariant::Input(poly_arg)
         } else if self.has_keyword(b"out") {
             self.consume_keyword(b"out");
             let poly_args = self.consume_polymorphic_args(h, allow_inference)?;
             let poly_arg = reduce_port_poly_args(h, &pos, poly_args)
                 .map_err(|| ParseError2::new_error(
                     &self.source, pos, "'out' type only accepts up to 1 polymorphic argument"
                 ))?;
             ParserTypeVariant::Output(poly_arg)
         } else {
             // Must be a symbolic type
             let identifier = self.consume_namespaced_identifier()?;
             let poly_args = self.consume_polymorphic_args(h, allow_inference)?;
             ParserTypeVariant::Symbolic(SymbolicParserType{identifier, poly_args, variant: None})
         };
         // If the type was a basic type (not supporting polymorphic type
         // arguments), then we make sure the user did not specify any of them.
         let mut backup_pos = self.source.pos();
         if !parser_type.supports_polymorphic_args() {
             self.consume_whitespace(false)?;
             if let Some(b'<') = self.source.next() {
                 return Err(ParseError2::new_error(
                     &self.source, self.source.pos(),
                     "This type does not allow polymorphic arguments"
                 ));
+            }
             self.source.seek(backup_pos);
+        }
         let mut parser_type_id = h.alloc_parser_type(|this| ParserType{
             this, pos, variant: parser_type_variant
         });
         // If we're dealing with arrays, then we need to wrap the currently
         // parsed type in array types
         self.consume_whitespace(false)?;
         while let Some(b'[') = self.source.next() {
             let pos = self.source.pos();
             self.source.consume();
             self.consume_whitespace(false)?;
             if let Some(b']') = self.source.next() {
                 // Type is wrapped in an array
                 self.source.consume();
                 parser_type_id = h.alloc_parser_type(|this| ParserType{
                     this, pos, variant: ParserTypeVariant::Array(parser_type_id)
                 });
                 backup_pos = self.source.pos();
                 // In case we're dealing with another array
                 self.consume_whitespace(false)?;
             } else {
                 return Err(ParseError2::new_error(
                     &self.source, pos,
                     "Expected a closing ']'"
                 ));
+            }
+        }
         self.source.seek(backup_pos);
         Ok(parser_type_id)
+    }
     /// Consumes polymorphic arguments and its delimiters if specified. The
     /// input position may be at whitespace. If polyargs are present then the
     /// whitespace and the args are consumed and the input position will be
     /// placed after the polyarg list. If polyargs are not present then the
     /// input position will remain unmodified and an empty vector will be
     /// returned.
     ///
     /// Polymorphic arguments represent the specification of the parametric
     /// types of a polymorphic type: they specify the value of the polymorphic
     /// type's polymorphic variables.
     fn consume_polymorphic_args(&mut self, h: &mut Heap, allow_inference: bool) -> Result<Vec<ParserTypeId>, ParseError2> {
         let backup_pos = self.source.pos();
         self.consume_whitespace(false)?;
         if let Some(b'<') = self.source.next() {
             // Has polymorphic args, at least one type must be specified
             self.source.consume();
             self.consume_whitespace(false)?;
             let mut poly_args = Vec::new();
             loop {
                 // TODO: @cleanup, remove the no_more_types var
                 poly_args.push(self.consume_type2(h, allow_inference)?);
                 self.consume_whitespace(false)?;
                 let has_comma = self.source.next() == Some(b',');
                 if has_comma {
                     // We might not actually be getting more types when the
                     // comma is at the end of the line, and we get a closing
                     // angular bracket on the next line.
                     self.source.consume();
                     self.consume_whitespace(false)?;
+                }
                 if let Some(b'>') = self.source.next() {
                     self.source.consume();
                     break;
                 } else if !has_comma {
                     return Err(ParseError2::new_error(
                         &self.source, self.source.pos(),
                         "Expected the end of the polymorphic argument list"
                     ))
+                }
+            }
             Ok(poly_args)
         } else {
             // No polymorphic args
             self.source.seek(backup_pos);
             Ok(vec!())
+        }
+    }
     /// Consumes polymorphic variables. These are identifiers that are used
     /// within polymorphic types. The input position may be at whitespace. If
     /// polymorphic variables are present then the whitespace, wrapping
     /// delimiters and the polymorphic variables are consumed. Otherwise the
     /// input position will stay where it is. If no polymorphic variables are
     /// present then an empty vector will be returned.
     fn consume_polymorphic_vars(&mut self) -> Result<Vec<Identifier>, ParseError2> {
         let backup_pos = self.source.pos();
         self.consume_whitespace(false)?;
         if let Some(b'<') = self.source.next() {
             // Found the opening delimiter, we want at least one polyvar
             self.source.consume();
             self.consume_whitespace(false)?;
             let mut poly_vars = Vec::new();
             loop {
                 poly_vars.push(self.consume_identifier()?);
                 self.consume_whitespace(false)?;
                 let has_comma = self.source.next() == Some(b',');
                 if has_comma {
                     // We may get another variable
                     self.source.consume();
                     self.consume_whitespace(false)?;
+                }
                 if let Some(b'>') = self.source.next() {
                     self.source.consume();
                     break;
                 } else if !has_comma {
                     return Err(ParseError2::new_error(
                         &self.source, self.source.pos(),
                         "Expected the end of the polymorphic variable list"
                     ))
+                }
+            }
             Ok(poly_vars)
         } else {
             // No polymorphic args
             self.source.seek(backup_pos);
             Ok(vec!())
+        }
+    }
     fn consume_primitive_type(&mut self) -> Result<PrimitiveType, ParseError2> {
         if self.has_keyword(b"in") {
             self.consume_keyword(b"in")?;
             Ok(PrimitiveType::Input)
         } else if self.has_keyword(b"out") {
             self.consume_keyword(b"out")?;
             Ok(PrimitiveType::Output)
         } else if self.has_keyword(b"msg") {
             self.consume_keyword(b"msg")?;
             Ok(PrimitiveType::Message)
         } else if self.has_keyword(b"boolean") {
             self.consume_keyword(b"boolean")?;
             Ok(PrimitiveType::Boolean)
         } else if self.has_keyword(b"byte") {
             self.consume_keyword(b"byte")?;
             Ok(PrimitiveType::Byte)
         } else if self.has_keyword(b"short") {
             self.consume_keyword(b"short")?;
             Ok(PrimitiveType::Short)
         } else if self.has_keyword(b"int") {
             self.consume_keyword(b"int")?;
             Ok(PrimitiveType::Int)
         } else if self.has_keyword(b"long") {
             self.consume_keyword(b"long")?;
             Ok(PrimitiveType::Long)
-        } else if self.has_keyword(b"let") {
+        } else if self.has_keyword(b"auto") {
             // TODO: @types
-            return Err(self.error_at_pos("inferred types using 'let' are reserved, but not yet implemented"));
+            return Err(self.error_at_pos("inferred types using 'auto' are reserved, but not yet implemented"));
         } else {
             let identifier = self.consume_namespaced_identifier()?;
             Ok(PrimitiveType::Symbolic(PrimitiveSymbolic{
                 identifier,
                 definition: None
             }))
+        }
+    }
     fn has_array(&mut self) -> bool {
         let backup_pos = self.source.pos();
         let mut result = false;
         match self.consume_whitespace(false) {
             Ok(_) => result = self.has_string(b"["),
             Err(_) => {}
+        }
         self.source.seek(backup_pos);
         return result;
+    }
     fn consume_type(&mut self) -> Result<Type, ParseError2> {
         let primitive = self.consume_primitive_type()?;
         let array;
         if self.has_array() {
             self.consume_string(b"[]")?;
             array = true;
         } else {
             array = false;
+        }
         Ok(Type { primitive, array })
+    }
     fn create_type_annotation_input(&self, h: &mut Heap) -> Result<TypeAnnotationId, ParseError2> {
         let position = self.source.pos();
         let the_type = Type::INPUT;
         let id = h.alloc_type_annotation(|this| TypeAnnotation { this, position, the_type });
         Ok(id)
+    }
     fn create_type_annotation_output(&self, h: &mut Heap) -> Result<TypeAnnotationId, ParseError2> {
         let position = self.source.pos();
         let the_type = Type::OUTPUT;
         let id = h.alloc_type_annotation(|this| TypeAnnotation { this, position, the_type });
         Ok(id)
+    }
     fn consume_type_annotation(&mut self, h: &mut Heap) -> Result<TypeAnnotationId, ParseError2> {
         let position = self.source.pos();
         let the_type = self.consume_type()?;
         let id = h.alloc_type_annotation(|this| TypeAnnotation { this, position, the_type });
         Ok(id)
+    }
     fn consume_type_annotation_spilled(&mut self) -> Result<(), ParseError2> {
         self.consume_type()?;
         Ok(())
+    }
     // Parameters
     fn consume_parameter(&mut self, h: &mut Heap) -> Result<ParameterId, ParseError2> {
         let position = self.source.pos();
         let type_annotation = self.consume_type_annotation(h)?;
         let parser_type = self.consume_type2(h, false)?;
         self.consume_whitespace(true)?;
         let position = self.source.pos();
         let identifier = self.consume_identifier()?;
         let id =
-            h.alloc_parameter(|this| Parameter { this, position, type_annotation, identifier });
+            h.alloc_parameter(|this| Parameter { this, position, parser_type, identifier });
         Ok(id)
+    }
     fn consume_parameters(
         &mut self,
         h: &mut Heap,
         params: &mut Vec<ParameterId>,
     ) -> Result<(), ParseError2> {
         self.consume_string(b"(")?;
         self.consume_whitespace(false)?;
         if !self.has_string(b")") {
             while self.source.next().is_some() {
                 params.push(self.consume_parameter(h)?);
                 self.consume_whitespace(false)?;
                 if self.has_string(b")") {
                     break;
+                }
                 self.consume_string(b",")?;
                 self.consume_whitespace(false)?;
+            }
+        }
         self.consume_string(b")")?;
         Ok(())
+    }
     // ====================
     // Expressions
     // ====================
     fn consume_paren_expression(&mut self, h: &mut Heap) -> Result<ExpressionId, ParseError2> {
         self.consume_string(b"(")?;
         self.consume_whitespace(false)?;
         let result = self.consume_expression(h)?;
         self.consume_whitespace(false)?;
         self.consume_string(b")")?;
         Ok(result)
+    }
     fn consume_expression(&mut self, h: &mut Heap) -> Result<ExpressionId, ParseError2> {
         if self.level >= MAX_LEVEL {
             return Err(self.error_at_pos("Too deeply nested expression"));
+        }
         self.level += 1;
         let result = self.consume_assignment_expression(h);
         self.level -= 1;
         result
+    }
     fn consume_assignment_expression(&mut self, h: &mut Heap) -> Result<ExpressionId, ParseError2> {
         let result = self.consume_conditional_expression(h)?;
@@ @@ -1305,144 +1531,168 @@ impl Lexer<'_> { @@
         if let Ok(_) = self.consume_type_annotation_spilled() {
             if let Ok(_) = self.consume_whitespace(false) {
                 result = self.has_identifier();
+            }
+        }
         self.source.seek(backup_pos);
         return result;
+    }
     fn consume_block_statement(&mut self, h: &mut Heap) -> Result<StatementId, ParseError2> {
         let position = self.source.pos();
         let mut statements = Vec::new();
         self.consume_string(b"{")?;
         self.consume_whitespace(false)?;
         while self.has_local_statement() {
             statements.push(self.consume_local_statement(h)?.upcast());
             self.consume_whitespace(false)?;
+        }
         while !self.has_string(b"}") {
             statements.push(self.consume_statement(h, false)?);
             self.consume_whitespace(false)?;
+        }
         self.consume_string(b"}")?;
         if statements.is_empty() {
             Ok(h.alloc_skip_statement(|this| SkipStatement { this, position, next: None }).upcast())
         } else {
             Ok(h.alloc_block_statement(|this| BlockStatement {
                 this,
                 position,
                 statements,
                 parent_scope: None,
                 relative_pos_in_parent: 0,
                 locals: Vec::new(),
                 labels: Vec::new(),
             })
             .upcast())
+        }
+    }
     fn consume_local_statement(&mut self, h: &mut Heap) -> Result<LocalStatementId, ParseError2> {
         if self.has_keyword(b"channel") {
             Ok(self.consume_channel_statement(h)?.upcast())
         } else {
             Ok(self.consume_memory_statement(h)?.upcast())
+        }
+    }
     fn consume_channel_statement(
         &mut self,
         h: &mut Heap,
     ) -> Result<ChannelStatementId, ParseError2> {
         // Consume channel statement and polymorphic argument if specified
         let position = self.source.pos();
         self.consume_keyword(b"channel")?;
         let poly_args = self.consume_polymorphic_args(h, true)?;
         let poly_arg_id = match poly_args.len() {
 => h.alloc_parser_type(|this| ParserType{
                 this, pos: position.clone(), variant: ParserTypeVariant::Inferred,
             }),
 => poly_args[0],
             _ => return Err(ParseError2::new_error(
                 &self.source, self.source.pos(),
                 "port construction using 'channel' accepts up to 1 polymorphic argument"
             ))
         };
         self.consume_whitespace(true)?;
         let from_annotation = self.create_type_annotation_output(h)?;
         let from_identifier = self.consume_identifier()?;
         // Consume the output port
         let out_parser_type = h.alloc_parser_type(|this| ParserType{
             this, pos: position.clone(), variant: ParserTypeVariant::Output(poly_arg_id)
         });
         let out_identifier = self.consume_identifier()?;
         // Consume the "->" syntax
         self.consume_whitespace(false)?;
         self.consume_string(b"->")?;
         self.consume_whitespace(false)?;
         let to_annotation = self.create_type_annotation_input(h)?;
         let to_identifier = self.consume_identifier()?;
         // Consume the input port
         // TODO: Unsure about this, both ports refer to the same ParserType, is this ok?
         let in_parser_type = h.alloc_parser_type(|this| ParserType{
             this, pos: position.clone(), variant: ParserTypeVariant::Output(poly_arg_id)
         });
         let in_identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
-        let from = h.alloc_local(|this| Local {
+        let out_port = h.alloc_local(|this| Local {
             this,
             position,
             type_annotation: from_annotation,
             identifier: from_identifier,
             parser_type: out_parser_type,
             identifier: out_identifier,
             relative_pos_in_block: 0
         });
-        let to = h.alloc_local(|this| Local {
+        let in_port = h.alloc_local(|this| Local {
             this,
             position,
             type_annotation: to_annotation,
             identifier: to_identifier,
             parser_type: in_parser_type,
             identifier: in_identifier,
             relative_pos_in_block: 0
         });
         Ok(h.alloc_channel_statement(|this| ChannelStatement {
             this,
             position,
             from,
             to,
             from: out_port,
             to: in_port,
             relative_pos_in_block: 0,
             next: None,
         }))
+    }
     fn consume_memory_statement(&mut self, h: &mut Heap) -> Result<MemoryStatementId, ParseError2> {
         let position = self.source.pos();
-        let type_annotation = self.consume_type_annotation(h)?;
+        let parser_type = self.consume_type2(h, true)?;
         self.consume_whitespace(true)?;
         let identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b"=")?;
         self.consume_whitespace(false)?;
         let initial = self.consume_expression(h)?;
         let variable = h.alloc_local(|this| Local {
             this,
             position,
-            type_annotation,
+            parser_type,
             identifier,
             relative_pos_in_block: 0
         });
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_memory_statement(|this| MemoryStatement {
             this,
             position,
             variable,
             initial,
             next: None,
         }))
+    }
     fn consume_labeled_statement(
         &mut self,
         h: &mut Heap,
     ) -> Result<LabeledStatementId, ParseError2> {
         let position = self.source.pos();
         let label = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b":")?;
         self.consume_whitespace(false)?;
         let body = self.consume_statement(h, false)?;
         Ok(h.alloc_labeled_statement(|this| LabeledStatement {
             this,
             position,
             label,
             body,
             relative_pos_in_block: 0,
             in_sync: None,
         }))
+    }
     fn consume_skip_statement(&mut self, h: &mut Heap) -> Result<SkipStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"skip")?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_skip_statement(|this| SkipStatement { this, position, next: None }))
+    }
     fn consume_if_statement(&mut self, h: &mut Heap) -> Result<IfStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"if")?;
         self.consume_whitespace(false)?;
         let test = self.consume_paren_expression(h)?;
         self.consume_whitespace(false)?;
         let true_body = self.consume_statement(h, true)?;
         self.consume_whitespace(false)?;
         let false_body = if self.has_keyword(b"else") {
@@ @@ -1588,286 +1838,321 @@ impl Lexer<'_> { @@
         self.consume_whitespace(false)?;
         self.consume_string(b",")?;
         self.consume_whitespace(false)?;
         let message = self.consume_expression(h)?;
         self.consume_whitespace(false)?;
         self.consume_string(b")")?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_put_statement(|this| PutStatement { this, position, port, message, next: None }))
+    }
     fn consume_expression_statement(
         &mut self,
         h: &mut Heap,
     ) -> Result<ExpressionStatementId, ParseError2> {
         let position = self.source.pos();
         let expression = self.consume_expression(h)?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_expression_statement(|this| ExpressionStatement {
             this,
             position,
             expression,
             next: None,
         }))
+    }
     // ====================
     // Symbol definitions
     // ====================
     fn has_symbol_definition(&self) -> bool {
         self.has_keyword(b"composite")
             || self.has_keyword(b"primitive")
             || self.has_type_keyword()
             || self.has_identifier()
+    }
     fn consume_symbol_definition(&mut self, h: &mut Heap) -> Result<DefinitionId, ParseError2> {
         if self.has_keyword(b"struct") {
             Ok(self.consume_struct_definition(h)?.upcast())
         } else if self.has_keyword(b"enum") {
             Ok(self.consume_enum_definition(h)?.upcast())
         } else if self.has_keyword(b"composite") || self.has_keyword(b"primitive") {
             Ok(self.consume_component_definition(h)?.upcast())
         } else {
             Ok(self.consume_function_definition(h)?.upcast())
+        }
+    }
     fn consume_struct_definition(&mut self, h: &mut Heap) -> Result<StructId, ParseError2> {
         // Parse "struct" keyword and its identifier
+        // Parse "struct" keyword, optional polyvars and its identifier
         let struct_pos = self.source.pos();
         self.consume_keyword(b"struct")?;
         self.consume_whitespace(true)?;
         let struct_ident = self.consume_identifier()?;
         let poly_vars = self.consume_polymorphic_vars()?;
         self.consume_whitespace(false)?;
         // Parse struct fields
         self.consume_string(b"{")?;
         let mut next = self.source.next();
         let mut fields = Vec::new();
         while next.is_some() {
             let char = next.unwrap();
             if char == b'}' {
                 break;
+            }
             // Consume field definition
             self.consume_whitespace(false)?;
             let field_position = self.source.pos();
-            let field_type = self.consume_type_annotation(h)?;
+            let field_parser_type = self.consume_type2(h, false)?;
             self.consume_whitespace(true)?;
             let field_ident = self.consume_identifier()?;
             self.consume_whitespace(false)?;
             fields.push(StructFieldDefinition{
                 position: field_position,
                 field: field_ident,
-                the_type: field_type,
+                parser_type: field_parser_type,
             });
             // If we have a comma, then we may or may not have another field
             // definition. Otherwise we expect the struct to be fully defined
             // and expect a closing brace
             next = self.source.next();
             if let Some(b',') = next {
                 self.source.consume();
                 self.consume_whitespace(false)?;
                 next = self.source.next();
             } else {
                 break;
+            }
+        }
         // End of struct definition, so we expect a closing brace
         self.consume_string(b"}")?;
         // Valid struct definition
         Ok(h.alloc_struct_definition(|this| StructDefinition{
             this,
             position: struct_pos,
             identifier: struct_ident,
             poly_vars,
             fields,
         }))
+    }
     fn consume_enum_definition(&mut self, h: &mut Heap) -> Result<EnumId, ParseError2> {
         // Parse "enum" keyword and its identifier
+        // Parse "enum" keyword, optional polyvars and its identifier
         let enum_pos = self.source.pos();
         self.consume_keyword(b"enum")?;
         self.consume_whitespace(true)?;
         let enum_ident = self.consume_identifier()?;
         let poly_vars = self.consume_polymorphic_vars()?;
         self.consume_whitespace(false)?;
         // Parse enum variants
         self.consume_string(b"{")?;
         let mut next = self.source.next();
         let mut variants = Vec::new();
         while next.is_some() {
             let char = next.unwrap();
             if char == b'}' {
                 break;
+            }
             // Consume variant identifier
             self.consume_whitespace(false)?;
             let variant_position = self.source.pos();
             let variant_ident = self.consume_identifier()?;
             self.consume_whitespace(false)?;
             // Consume variant (tag) value: may be nothing, in which case it is
             // assigned automatically, may be a constant integer, or an embedded
             // type as value, resulting in a tagged union
             next = self.source.next();
             let variant_value = if let Some(b',') = next {
                 EnumVariantValue::None
             } else if let Some(b'=') = next {
                 self.source.consume();
                 self.consume_whitespace(false)?;
                 if !self.has_integer() {
                     return Err(self.error_at_pos("expected integer"));
+                }
                 let variant_int = self.consume_integer()?;
                 self.consume_whitespace(false)?;
                 EnumVariantValue::Integer(variant_int)
             } else if let Some(b'(') = next {
                 self.source.consume();
                 self.consume_whitespace(false)?;
-                let variant_type = self.consume_type_annotation(h)?;
+                let variant_type = self.consume_type2(h, false)?;
                 self.consume_whitespace(false)?;
                 self.consume_string(b")")?;
                 self.consume_whitespace(false)?;
                 EnumVariantValue::Type(variant_type)
             } else {
                 return Err(self.error_at_pos("expected ',', '=', or '('"));
             };
             variants.push(EnumVariantDefinition{
                 position: variant_position,
                 identifier: variant_ident,
                 value: variant_value
             });
             // If we have a comma, then we may or may not have another variant,
             // otherwise we expect the enum is fully defined
             next = self.source.next();
             if let Some(b',') = next {
                 self.source.consume();
                 self.consume_whitespace(false)?;
                 next = self.source.next();
             } else {
                 break;
+            }
+        }
         self.consume_string(b"}")?;
         // An enum without variants is somewhat valid, but completely useless
         // within the language
         if variants.is_empty() {
             return Err(ParseError2::new_error(self.source, enum_pos, "enum definition without variants"));
+        }
         Ok(h.alloc_enum_definition(|this| EnumDefinition{
             this,
             position: enum_pos,
             identifier: enum_ident,
             poly_vars,
             variants,
         }))
+    }
     fn consume_component_definition(&mut self, h: &mut Heap) -> Result<ComponentId, ParseError2> {
         // TODO: Cleanup
         if self.has_keyword(b"composite") {
             Ok(self.consume_composite_definition(h)?)
         } else {
             Ok(self.consume_primitive_definition(h)?)
+        }
+    }
     fn consume_composite_definition(&mut self, h: &mut Heap) -> Result<ComponentId, ParseError2> {
         // Parse keyword, optional polyvars and the identifier
         let position = self.source.pos();
         self.consume_keyword(b"composite")?;
         self.consume_whitespace(true)?;
         let identifier = self.consume_identifier()?;
         let poly_vars = self.consume_polymorphic_vars()?;
         self.consume_whitespace(false)?;
         // Consume parameters
         let mut parameters = Vec::new();
         self.consume_parameters(h, &mut parameters)?;
         self.consume_whitespace(false)?;
         // Parse body
         let body = self.consume_block_statement(h)?;
         Ok(h.alloc_component(|this| Component {
             this, variant: ComponentVariant::Composite, position, identifier, parameters, body
             this,
             variant: ComponentVariant::Composite,
             position,
             identifier,
             poly_vars,
             parameters,
             body
         }))
+    }
     fn consume_primitive_definition(&mut self, h: &mut Heap) -> Result<ComponentId, ParseError2> {
         // Consume keyword, optional polyvars and identifier
         let position = self.source.pos();
         self.consume_keyword(b"primitive")?;
         self.consume_whitespace(true)?;
         let identifier = self.consume_identifier()?;
         let poly_vars = self.consume_polymorphic_vars()?;
         self.consume_whitespace(false)?;
         // Consume parameters
         let mut parameters = Vec::new();
         self.consume_parameters(h, &mut parameters)?;
         self.consume_whitespace(false)?;
         // Consume body
         let body = self.consume_block_statement(h)?;
         Ok(h.alloc_component(|this| Component {
             this, variant: ComponentVariant::Primitive, position, identifier, parameters, body
             this,
             variant: ComponentVariant::Primitive,
             position,
             identifier,
             poly_vars,
             parameters,
             body
         }))
+    }
     fn consume_function_definition(&mut self, h: &mut Heap) -> Result<FunctionId, ParseError2> {
         // Consume return type, optional polyvars and identifier
         let position = self.source.pos();
         let return_type = self.consume_type_annotation(h)?;
         self.consume_whitespace(true)?;
         let identifier = self.consume_identifier()?;
         let poly_vars = self.consume_polymorphic_vars()?;
         self.consume_whitespace(false)?;
         // Consume parameters
         let mut parameters = Vec::new();
         self.consume_parameters(h, &mut parameters)?;
         self.consume_whitespace(false)?;
         // Consume body
         let body = self.consume_block_statement(h)?;
         Ok(h.alloc_function(|this| Function {
             this,
             position,
             return_type,
             identifier,
             poly_vars,
             parameters,
             body,
         }))
+    }
     fn has_pragma(&self) -> bool {
         if let Some(c) = self.source.next() {
             c == b'#'
         } else {
             false
+        }
+    }
     fn consume_pragma(&mut self, h: &mut Heap) -> Result<PragmaId, ParseError2> {
         let position = self.source.pos();
         let next = self.source.next();
         if next != Some(b'#') {
             return Err(self.error_at_pos("Expected pragma"));
+        }
         self.source.consume();
         if !is_vchar(self.source.next()) {
             return Err(self.error_at_pos("Expected pragma"));
+        }
         if self.has_string(b"version") {
             self.consume_string(b"version")?;
             self.consume_whitespace(true)?;
             if !self.has_integer() {
                 return Err(self.error_at_pos("Expected integer constant"));
+            }
             let version = self.consume_integer()?;
             debug_assert!(version >= 0);
             return Ok(h.alloc_pragma(|this| Pragma::Version(PragmaVersion{
                 this, position, version: version as u64
             })))
         } else if self.has_string(b"module") {
             self.consume_string(b"module")?;
             self.consume_whitespace(true)?;
             if !self.has_identifier() {
                 return Err(self.error_at_pos("Expected identifier"));
+            }
             let mut value = Vec::new();
             let mut ident = self.consume_ident()?;
             value.append(&mut ident);
             while self.has_string(b".") {
                 self.consume_string(b".")?;
                 value.push(b'.');
                 ident = self.consume_ident()?;
                 value.append(&mut ident);
+            }
             return Ok(h.alloc_pragma(|this| Pragma::Module(PragmaModule{

src/protocol/parser/mod.rs

➞

Show inline comments

 mod depth_visitor;
 mod symbol_table;
 mod type_table;
 mod type_table2;
 mod type_resolver;
 mod visitor;
 mod visitor_linker;
 use depth_visitor::*;
 use symbol_table::SymbolTable;
 use visitor::Visitor2;
 use visitor_linker::ValidityAndLinkerVisitor;
 use type_table::TypeTable;
 use crate::protocol::ast::*;
 use crate::protocol::inputsource::*;
 use crate::protocol::lexer::*;
 use std::collections::HashMap;
 use crate::protocol::ast_printer::ASTWriter;
 // TODO: @fixme, pub qualifier
 pub(crate) struct LexedModule {
     pub(crate) source: InputSource,
     module_name: Vec<u8>,
     version: Option<u64>,
     root_id: RootId,
+}
 pub struct Parser {
     pub(crate) heap: Heap,
     pub(crate) modules: Vec<LexedModule>,
     pub(crate) module_lookup: HashMap<Vec<u8>, usize>, // from (optional) module name to `modules` idx
+}
 impl Parser {
     pub fn new() -> Self {
         Parser{
             heap: Heap::new(),
             modules: Vec::new(),
             module_lookup: HashMap::new()
+        }
+    }
     // TODO: @fix, temporary implementation to keep code compilable
     pub fn new_with_source(source: InputSource) -> Result<Self, ParseError2> {
         let mut parser = Parser::new();
         parser.feed(source)?;
         Ok(parser)
+    }
     pub fn feed(&mut self, mut source: InputSource) -> Result<RootId, ParseError2> {

src/protocol/parser/symbol_table.rs

➞

Show inline comments

 // TODO: Maybe allow namespaced-aliased imports. It is currently not possible
 //  to express the following:
 //      import Module.Submodule as SubMod
 //      import SubMod::{Symbol}
 //  And it is especially not possible to express the following:
 //      import SubMod::{Symbol}
 //      import Module.Submodule as SubMod
 use crate::protocol::ast::*;
 use crate::protocol::inputsource::*;
 use std::collections::{HashMap, hash_map::Entry};
 use crate::protocol::parser::LexedModule;
 #[derive(PartialEq, Eq, Hash)]
 struct SymbolKey {
     module_id: RootId,
     symbol_name: Vec<u8>,
+}
 pub(crate) enum Symbol {
     Namespace(RootId),
     Definition((RootId, DefinitionId)),
+}
 pub(crate) struct SymbolValue {
     // Position refers to the origin of the symbol definition (i.e. the module's
     // RootId that is in the key being used to lookup this value)
     // Position is the place where the symbol is introduced to a module (this
     // position always corresponds to the module whose RootId is stored in the
     // `SymbolKey` associated with this `SymbolValue`). For a definition this
     // is the position where the symbol is defined, for an import this is the
     // position of the import statement.
     pub(crate) position: InputPosition,
     pub(crate) symbol: Symbol,
+}
 impl SymbolValue {
     pub(crate) fn is_namespace(&self) -> bool {
         match &self.symbol {
             Symbol::Namespace(_) => true,
             _ => false
+        }
+    }
     pub(crate) fn as_namespace(&self) -> Option<RootId> {
         match &self.symbol {
             Symbol::Namespace(root_id) => Some(*root_id),
             _ => None,
+        }
+    }
     pub(crate) fn as_definition(&self) -> Option<(RootId, DefinitionId)> {
         match &self.symbol {
             Symbol::Definition((root_id, definition_id)) => Some((*root_id, *definition_id)),
             _ => None,
+        }
+    }
+}
 /// `SymbolTable` is responsible for two parts of the parsing process: firstly
 /// it ensures that there are no clashing symbol definitions within each file,
 /// and secondly it will resolve all symbols within a module to their
 /// appropriate definitions (in case of enums, functions, etc.) and namespaces
 /// (currently only external modules can act as namespaces). If a symbol clashes
 /// or if a symbol cannot be resolved this will be an error.
 ///
 /// Within the compilation process the symbol table is responsible for resolving
 /// namespaced identifiers (e.g. Module::Enum::EnumVariant) to the appropriate
 /// definition (i.e. not namespaces; as the language has no way to use
 /// namespaces except for using them in namespaced identifiers).
 pub(crate) struct SymbolTable {
     // Lookup from module name (not any aliases) to the root id
     module_lookup: HashMap<Vec<u8>, RootId>,
     // Lookup from within a module, to a particular imported (potentially
     // aliased) or defined symbol. Basically speaking: if the source code of a
     // module contains correctly imported/defined symbols, then this lookup
     // will always return the corresponding definition
     symbol_lookup: HashMap<SymbolKey, SymbolValue>,
+}
 impl SymbolTable {
     pub(crate) fn new(heap: &Heap, modules: &[LexedModule]) -> Result<Self, ParseError2> {

src/protocol/parser/type_table.rs

➞

Show inline comments

@@ @@ -31,96 +31,99 @@ impl TypeClass { @@
 impl std::fmt::Display for TypeClass {
     fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
         write!(f, "{}", self.display_name())
+    }
+}
 pub enum DefinedType {
     Enum(EnumType),
     Union(UnionType),
     Struct(StructType),
     Function(FunctionType),
     Component(ComponentType)
+}
 impl DefinedType {
     pub(crate) fn type_class(&self) -> TypeClass {
         match self {
             DefinedType::Enum(_) => TypeClass::Enum,
             DefinedType::Union(_) => TypeClass::Union,
             DefinedType::Struct(_) => TypeClass::Struct,
             DefinedType::Function(_) => TypeClass::Function,
             DefinedType::Component(_) => TypeClass::Component
+        }
+    }
+}
 // TODO: Also support maximum u64 value
 pub struct EnumVariant {
     identifier: Identifier,
     value: i64,
+}
 /// `EnumType` is the classical C/C++ enum type. It has various variants with
 /// an assigned integer value. The integer values may be user-defined,
 /// compiler-defined, or a mix of the two. If a user assigns the same enum
 /// value multiple times, we assume the user is an expert and we consider both
 /// variants to be equal to one another.
 pub struct EnumType {
     variants: Vec<EnumVariant>,
     representation: PrimitiveType,
+}
 pub struct UnionVariant {
     identifier: Identifier,
     embedded_type: Option<TypeAnnotationId>,
     tag_value: i64,
+}
 /// `UnionType` is the algebraic datatype (or sum type, or discriminated union).
 /// A value is an element of the union, identified by its tag, and may contain
 /// a single subtype.
 pub struct UnionType {
     variants: Vec<UnionVariant>,
     tag_representation: PrimitiveType
+}
 pub struct StructField {
     identifier: Identifier,
     field_type: TypeAnnotationId,
+}
 pub struct StructType {
     fields: Vec<StructField>,
+}
 pub struct FunctionArgument {
     identifier: Identifier,
     argument_type: TypeAnnotationId,
+}
 pub struct FunctionType {
     return_type: TypeAnnotationId,
     arguments: Vec<FunctionArgument>
+}
 pub struct ComponentType {
     variant: ComponentVariant,
     arguments: Vec<FunctionArgument>
+}
 pub struct TypeTable {
     lookup: HashMap<DefinitionId, DefinedType>,
+}
 enum LookupResult {
     BuiltIn,
     Resolved((RootId, DefinitionId)),
     Unresolved((RootId, DefinitionId)),
     Error((InputPosition, String)),
+}
 /// `TypeTable` is responsible for walking the entire lexed AST and laying out
 /// the various user-defined types in terms of bytes and offsets. This process
 /// may be pseudo-recursive (meaning: it is implemented in a recursive fashion,
 /// but not in the recursive-function-call kind of way) in case a type depends
 /// on another type to be resolved.
 /// TODO: Distinction between resolved types and unresolved types (regarding the
 ///     mixed use of BuiltIns and resolved types) is a bit yucky at the moment,
 ///     will have to come up with something nice in the future.
@@ @@ -227,96 +230,97 @@ impl TypeTable { @@
         let mut module_index = 0;
         let mut definition_index = 0;
         let mut breadcrumbs = Vec::with_capacity(32); // if a user exceeds this, the user sucks at programming
         while module_index < modules.len() {
             // Go to next module if needed
+            {
                 let root = &heap[modules[module_index].root_id];
                 if definition_index >= root.definitions.len() {
                     module_index += 1;
                     definition_index = 0;
                     continue;
+                }
+            }
             // Construct breadcrumbs in case we need to follow some types around
             debug_assert!(breadcrumbs.is_empty());
             breadcrumbs.push(Breadcrumb::Linear((module_index, definition_index)));
             'resolve_loop: while !breadcrumbs.is_empty() {
                 // Retrieve module, the module's root and the definition
                 let (module, definition_id) = match breadcrumbs.last().unwrap() {
                     Breadcrumb::Linear((module_index, definition_index)) => {
                         let module = &modules[*module_index];
                         let root = &heap[module.root_id];
                         let definition_id = root.definitions[*definition_index];
                         (module, definition_id)
                     },
                     Breadcrumb::Jumping((root_id, definition_id)) => {
                         let module = &modules[root_id.index as usize];
                         debug_assert_eq!(module.root_id, *root_id);
                         (module, *definition_id)
+                    }
                 };
                 let definition = &heap[definition_id];
                 // Because we might have chased around to this particular
                 // definition before, we check if we haven't resolved the type
                 // already.
                 if table.lookup.contains_key(&definition_id) {
                     breadcrumbs.pop();
                     continue;
+                }
                 match definition {
                     Definition::Enum(definition) => {
                         // Check the definition to see if we're dealing with an
                         // enum or a union. If we find any union variants then
                         // we immediately check if the type is already resolved.
                         assert!(definition.poly_vars.is_empty(), "Polyvars for enum not yet implemented");
                         let mut has_tag_values = None;
                         let mut has_int_values = None;
                         for variant in &definition.variants {
                             match &variant.value {
                                 EnumVariantValue::None => {},
                                 EnumVariantValue::Integer(_) => {
                                     if has_int_values.is_none() { has_int_values = Some(variant.position); }
                                  },
                                 EnumVariantValue::Type(variant_type) => {
                                     if has_tag_values.is_none() { has_tag_values = Some(variant.position); }
                                     let variant_type = &heap[*variant_type];
                                     let lookup_result = lookup_type_definition(
                                         heap, &table, symbols, module.root_id,
                                         &variant_type.the_type.primitive
                                     );
                                     if !handle_lookup_result(heap, modules, module, &mut breadcrumbs, lookup_result)? {
                                         continue 'resolve_loop;
+                                    }
                                 },
+                            }
+                        }
                         if has_tag_values.is_some() && has_int_values.is_some() {
                             // Not entirely illegal, but probably not desired
                             let tag_pos = has_tag_values.unwrap();
                             let int_pos = has_int_values.unwrap();
                             return Err(
                                 ParseError2::new_error(&module.source, definition.position, "Illegal combination of enum integer variant(s) and enum union variant(s)")
                                 .with_postfixed_info(&module.source, int_pos, "Explicitly assigning an integer value here")
                                 .with_postfixed_info(&module.source, tag_pos, "Explicitly declaring a union variant here")
+                            )
+                        }
                         // If here, then the definition is a valid discriminated
                         // union with all of its types resolved, or a valid
                         // enum.
                         // Decide whether to implement as enum or as union
                         let is_union = has_tag_values.is_some();
                         if is_union {
                             // Implement as discriminated union. Because we
                             // checked the availability of types above, we are
                             // safe to lookup type definitions
                             let mut tag_value = -1;
                             let mut variants = Vec::with_capacity(definition.variants.len());
                             for variant in &definition.variants {
@@ @@ -343,160 +347,163 @@ impl TypeTable { @@
                                 variants.push(UnionVariant{
                                     identifier: variant.identifier.clone(),
                                     embedded_type,
                                     tag_value,
                                 })
+                            }
                             table.add_definition(definition_id, DefinedType::Union(UnionType{
                                 variants,
                                 tag_representation: enum_representation(tag_value)
                             }));
                         } else {
                             // Implement as regular enum
                             let mut enum_value = -1; // TODO: allow u64 max size
                             let mut variants = Vec::with_capacity(definition.variants.len());
                             for variant in &definition.variants {
                                 enum_value += 1;
                                 match &variant.value {
                                     EnumVariantValue::None => {
                                         variants.push(EnumVariant{
                                             identifier: variant.identifier.clone(),
                                             value: enum_value,
                                         });
                                     },
                                     EnumVariantValue::Integer(override_value) => {
                                         enum_value = *override_value;
                                         variants.push(EnumVariant{
                                             identifier: variant.identifier.clone(),
                                             value: enum_value,
                                         });
                                     },
                                     EnumVariantValue::Type(_) => {
                                         debug_assert!(false, "Encountered `Type` variant after asserting enum is not a discriminated union");
                                         unreachable!();
+                                    }
+                                }
+                            }
                             table.add_definition(definition_id, DefinedType::Enum(EnumType{
                                 variants,
                                 representation: enum_representation(enum_value),
                             }));
+                        }
                     },
                     Definition::Struct(definition) => {
                         // Before we start allocating fields, make sure we can
                         // actually resolve all of the field types
                         assert!(definition.poly_vars.is_empty(), "Polyvars for struct not yet implemented");
                         for field_definition in &definition.fields {
                             let type_definition = &heap[field_definition.the_type];
                             let lookup_result = lookup_type_definition(
                                 heap, &table, symbols, module.root_id,
                                 &type_definition.the_type.primitive
                             );
                             if !handle_lookup_result(heap, modules, module, &mut breadcrumbs, lookup_result)? {
                                 continue 'resolve_loop;
+                            }
+                        }
                         // We can resolve everything
                         let mut fields = Vec::with_capacity(definition.fields.len());
                         for field_definition in &definition.fields {
                             let type_annotation = &heap[field_definition.the_type];
                             if cfg!(debug_assertions) {
                                 ensure_type_definition(heap, &table, symbols, module.root_id, &type_annotation.the_type.primitive);
+                            }
                             fields.push(StructField{
                                 identifier: field_definition.field.clone(),
                                 field_type: field_definition.the_type
                             });
+                        }
                         table.add_definition(definition_id, DefinedType::Struct(StructType{
                             fields,
                         }));
                     },
                     Definition::Component(definition) => {
                         // As always, ensure all parameter types are resolved
                         assert!(definition.poly_vars.is_empty(), "Polyvars for component not yet implemented");
                         for parameter_id in &definition.parameters {
                             let parameter = &heap[*parameter_id];
                             let type_definition = &heap[parameter.type_annotation];
                             let lookup_result = lookup_type_definition(
                                 heap, &table, symbols, module.root_id,
                                 &type_definition.the_type.primitive
                             );
                             if !handle_lookup_result(heap, modules, module, &mut breadcrumbs, lookup_result)? {
                                 continue 'resolve_loop;
+                            }
+                        }
                         // We can resolve everything
                         let mut parameters = Vec::with_capacity(definition.parameters.len());
                         for parameter_id in &definition.parameters {
                             let parameter = &heap[*parameter_id];
                             let type_definition = &heap[parameter.type_annotation];
                             if cfg!(debug_assertions) {
                                 ensure_type_definition(heap, &table, symbols, module.root_id, &type_definition.the_type.primitive);
+                            }
                             parameters.push(FunctionArgument{
                                 identifier: parameter.identifier.clone(),
                                 argument_type: parameter.type_annotation,
                             });
+                        }
                         table.add_definition(definition_id, DefinedType::Component(ComponentType{
                             variant: definition.variant,
                             arguments: parameters, // Arguments, parameters, tomayto, tomahto
                         }));
                     },
                     Definition::Function(definition) => {
                         assert!(definition.poly_vars.is_empty(), "Polyvars for function not yet implemented");
                         // Handle checking the return type
                         let lookup_result = lookup_type_definition(
                             heap, &table, symbols, module.root_id,
                             &heap[definition.return_type].the_type.primitive
                         );
                         if !handle_lookup_result(heap, modules, module, &mut breadcrumbs, lookup_result)? {
                             continue 'resolve_loop;
+                        }
                         for parameter_id in &definition.parameters {
                             let parameter = &heap[*parameter_id];
                             let type_definition = &heap[parameter.type_annotation];
                             let lookup_result = lookup_type_definition(
                                 heap, &table, symbols, module.root_id,
                                 &type_definition.the_type.primitive
                             );
                             if !handle_lookup_result(heap, modules, module, &mut breadcrumbs, lookup_result)? {
                                 continue 'resolve_loop;
+                            }
+                        }
                         // Resolve function's types
                         let mut parameters = Vec::with_capacity(definition.parameters.len());
                         for parameter_id in &definition.parameters {
                             let parameter = &heap[*parameter_id];
                             let type_definition = &heap[parameter.type_annotation];
                             if cfg!(debug_assertions) {
                                 ensure_type_definition(heap, &table, symbols, module.root_id, &type_definition.the_type.primitive);
+                            }
                             parameters.push(FunctionArgument{
                                 identifier: parameter.identifier.clone(),
                                 argument_type: parameter.type_annotation
                             });
+                        }
                         table.add_definition(definition_id, DefinedType::Function(FunctionType{
                             return_type: definition.return_type.clone(),
                             arguments: parameters,
                         }));
                     },
+                }
                 // If here, then we layed out the current type definition under
                 // investigation, so:
                 debug_assert!(!breadcrumbs.is_empty());
                 breadcrumbs.pop();
+            }

src/protocol/parser/type_table2.rs

➞

Show inline comments

@@ new file 100644 @@
 /**
 TypeTable
 Contains the type table: a datastructure that, when compilation succeeds,
 contains a concrete type definition for each AST type definition. In general
 terms the type table will go through the following phases during the compilation
 process:
 . The base type definitions are resolved after the parser phase has
         finished. This implies that the AST is fully constructed, but not yet
         annotated.
 . With the base type definitions resolved, the validation/linker phase will
         use the type table (together with the symbol table) to disambiguate
         terms (e.g. does an expression refer to a variable, an enum, a constant,
         etc.)
 . During the type checking/inference phase the type table is used to ensure
         that the AST contains valid use of types in expressions and statements.
         At the same time type inference will find concrete instantiations of
         polymorphic types, these will be stored in the type table as monomorphed
         instantiations of a generic type.
 . After type checking and inference (and possibly when constructing byte
         code) the type table will construct a type graph and solidify each
         non-polymorphic type and monomorphed instantiations of polymorphic types
         into concrete types.
 So a base type is defined by its (optionally polymorphic) representation in the
 AST. A concrete type replaces all polymorphic arguments with inferred types. A
 struct, enum or union may have polymorphic arguments but not actually be a
 polymorphic type. This happens when the polymorphic arguments are not used in
 the type definition itself.
 NOTE: for now a polymorphic definition of a function/component is illegal if the
     polymorphic arguments are not used in the arguments/return type. It should
     be legal, but we disallow it for now.
 TODO: Allow potentially cyclic datatypes and reject truly cyclic datatypes.
 TODO: Allow for the full potential of polymorphism
 TODO: Detect "true" polymorphism: for datatypes like structs/enum/unions this
     is simple. For functions we need to check the entire body. Do it here? Or
     do it somewhere else?
 TODO: Do we want to check fn argument collision here, or in validation phase?
 */
 use std::fmt::{Formatter, Result as FmtResult};
 use std::collections::HashMap;
 use crate::protocol::ast::*;
 use crate::protocol::parser::symbol_table::{SymbolTable, Symbol};
 use crate::protocol::inputsource::*;
 use crate::protocol::parser::*;
 use crate::protocol::parser::type_table2::ResolveResult::Resolved;
 //------------------------------------------------------------------------------
 // Defined Types
 //------------------------------------------------------------------------------
 #[derive(Copy, Clone, PartialEq, Eq)]
 pub enum TypeClass {
     Enum,
     Union,
     Struct,
     Function,
     Component
+}
 impl TypeClass {
     pub(crate) fn display_name(&self) -> &'static str {
         match self {
             TypeClass::Enum => "enum",
             TypeClass::Union => "enum",
             TypeClass::Struct => "struct",
             TypeClass::Function => "function",
             TypeClass::Component => "component",
+        }
+    }
+}
 impl std::fmt::Display for TypeClass {
     fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
         write!(f, "{}", self.display_name())
+    }
+}
 /// Struct wrapping around a potentially polymorphic type. If the type does not
 /// have any polymorphic arguments then it will not have any monomorphs and
 /// `is_polymorph` will be set to `false`. A type with polymorphic arguments
 /// only has `is_polymorph` set to `true` if the polymorphic arguments actually
 /// appear in the types associated types (function return argument, struct
 /// field, enum variant, etc.). Otherwise the polymorphic argument is just a
 /// marker and does not influence the bytesize of the type.
 pub struct DefinedType {
     ast_definition: DefinitionId,
     definition: DefinedTypeVariant,
     poly_args: Vec<PolyArg>,
     is_polymorph: bool,
     is_pointerlike: bool,
     monomorphs: Vec<u32>, // TODO: ?
+}
 pub enum DefinedTypeVariant {
     Enum(EnumType),
     Union(UnionType),
     Struct(StructType),
     Function(FunctionType),
     Component(ComponentType)
+}
 pub struct PolyArg {
     identifier: Identifier,
     is_phantom: bool,
+}
 impl DefinedTypeVariant {
     pub(crate) fn type_class(&self) -> TypeClass {
         match self {
             DefinedTypeVariant::Enum(_) => TypeClass::Enum,
             DefinedTypeVariant::Union(_) => TypeClass::Union,
             DefinedTypeVariant::Struct(_) => TypeClass::Struct,
             DefinedTypeVariant::Function(_) => TypeClass::Function,
             DefinedTypeVariant::Component(_) => TypeClass::Component
+        }
+    }
+}
 /// `EnumType` is the classical C/C++ enum type. It has various variants with
 /// an assigned integer value. The integer values may be user-defined,
 /// compiler-defined, or a mix of the two. If a user assigns the same enum
 /// value multiple times, we assume the user is an expert and we consider both
 /// variants to be equal to one another.
 pub struct EnumType {
     variants: Vec<EnumVariant>,
     representation: PrimitiveType,
+}
 // TODO: Also support maximum u64 value
 pub struct EnumVariant {
     identifier: Identifier,
     value: i64,
+}
 /// `UnionType` is the algebraic datatype (or sum type, or discriminated union).
 /// A value is an element of the union, identified by its tag, and may contain
 /// a single subtype.
 pub struct UnionType {
     variants: Vec<UnionVariant>,
     tag_representation: PrimitiveType
+}
 pub struct UnionVariant {
     identifier: Identifier,
     parser_type: Option<ParserTypeId>,
     tag_value: i64,
+}
 pub struct StructType {
     fields: Vec<StructField>,
+}
 pub struct StructField {
     identifier: Identifier,
     parser_type: ParserTypeId,
+}
 pub struct FunctionType {
     return_type: ParserTypeId,
     arguments: Vec<FunctionArgument>
+}
 pub struct ComponentType {
     variant: ComponentVariant,
     arguments: Vec<FunctionArgument>
+}
 pub struct FunctionArgument {
     identifier: Identifier,
     parser_type: ParserTypeId,
+}
 //------------------------------------------------------------------------------
 // Type table
 //------------------------------------------------------------------------------
 // TODO: @cleanup Do I really need this, doesn't make the code that much cleaner
 struct TypeIterator {
     breadcrumbs: Vec<(RootId, DefinitionId)>
+}
 impl TypeIterator {
     fn new() -> Self {
         Self{ breadcrumbs: Vec::with_capacity(32) }
+    }
     fn reset(&mut self, root_id: RootId, definition_id: DefinitionId) {
         self.breadcrumbs.clear();
         self.breadcrumbs.push((root_id, definition_id))
+    }
     fn push(&mut self, root_id: RootId, definition_id: DefinitionId) {
         self.breadcrumbs.push((root_id, definition_id));
+    }
     fn contains(&self, root_id: RootId, definition_id: DefinitionId) -> bool {
         for (stored_root_id, stored_definition_id) in self.breadcrumbs.iter() {
             if stored_root_id == root_id && stored_definition_id == definition_id { return true; }
+        }
         return false
+    }
     fn top(&self) -> Option<(RootId, DefinitionId)> {
         self.breadcrumbs.last().map(|(r, d)| (*r, *d))
+    }
     fn pop(&mut self) {
         debug_assert!(!self.breadcrumbs.is_empty());
         self.breadcrumbs.pop();
+    }
+}
 #[derive(PartialEq, Eq)]
 enum SpecifiedTypeVariant {
     // No subtypes
     Message,
     Bool,
     Byte,
     Short,
     Int,
     Long,
     String,
     // Always one subtype
     ArrayOf,
     InputOf,
     OutputOf,
     // Variable number of subtypes, depending on the polymorphic arguments on
     // the definition
     InstanceOf(DefinitionId, usize)
+}
 #[derive(Eq)]
 struct SpecifiedType {
     /// Definition ID, may not be enough as the type may be polymorphic
     definition: DefinitionId,
     /// The polymorphic types for the definition. These are encoded in a list,
     /// which we interpret as the depth-first serialization of the type tree.
     poly_vars: Vec<SpecifiedTypeVariant>
+}
 impl PartialEq for SpecifiedType {
     fn eq(&self, other: &Self) -> bool {
         // Should point to same definition and have the same polyvars
         if self.definition.index != other.definition.index { return false; }
         if self.poly_vars.len() != other.poly_vars.len() { return false; }
         for (my_var, other_var) in self.poly_vars.iter().zip(other.poly_vars.iter()) {
             if my_var != other_var { return false; }
+        }
         return true
+    }
+}
 impl SpecifiedType {
     fn new_non_polymorph(definition: DefinitionId) -> Self {
         Self{ definition, poly_vars: Vec::new() }
+    }
     fn new_polymorph(definition: DefinitionId, heap: &Heap, parser_type_id: ParserTypeId) -> Self {
         // Serialize into concrete types
         let mut poly_vars = Vec::new();
         Self::construct_poly_vars(&mut poly_vars, heap, parser_type_id);
         Self{ definition, poly_vars }
+    }
     fn construct_poly_vars(poly_vars: &mut Vec<SpecifiedTypeVariant>, heap: &Heap, parser_type_id: ParserTypeId) {
         // Depth-first construction of poly vars
         let parser_type = &heap[parser_type_id];
         match &parser_type.variant {
             ParserTypeVariant::Message => { poly_vars.push(SpecifiedTypeVariant::Message); },
             ParserTypeVariant::Bool => { poly_vars.push(SpecifiedTypeVariant::Bool); },
             ParserTypeVariant::Byte => { poly_vars.push(SpecifiedTypeVariant::Byte); },
             ParserTypeVariant::Short => { poly_vars.push(SpecifiedTypeVariant::Short); },
             ParserTypeVariant::Int => { poly_vars.push(SpecifiedTypeVariant::Int); },
             ParserTypeVariant::Long => { poly_vars.push(SpecifiedTypeVariant::Long); },
             ParserTypeVariant::String => { poly_vars.push(SpecifiedTypeVariant::String); },
             ParserTypeVariant::Array(subtype_id) => {
                 poly_vars.push(SpecifiedTypeVariant::ArrayOf);
                 Self::construct_poly_vars(poly_vars, heap, *subtype_id);
             },
             ParserTypeVariant::Input(subtype_id) => {
                 poly_vars.push(SpecifiedTypeVariant::InputOf);
                 Self::construct_poly_vars(poly_vars, heap, *subtype_id);
             },
             ParserTypeVariant::Output(subtype_id) => {
                 poly_vars.push(SpecifiedTypeVariant::OutputOf);
                 Self::construct_poly_vars(poly_vars, heap, *subtype_id);
             },
             ParserTypeVariant::Symbolic(symbolic) => {
                 let definition_id = match symbolic.variant {
                     SymbolicParserTypeVariant::Definition(definition_id) => definition_id,
                     SymbolicParserTypeVariant::PolyArg(_) => {
                         // When construct entries in the type table, we no longer allow the
                         // unspecified types in the AST, we expect them to be fully inferred.
                         debug_assert!(false, "Encountered 'PolyArg' symbolic type. Expected fully inferred types");
                         unreachable!();
+                    }
                 };
                 poly_vars.push(SpecifiedTypeVariant::InstanceOf(definition_id, symbolic.poly_args.len()));
                 for subtype_id in &symbolic.poly_args {
                     Self::construct_poly_vars(poly_vars, heap, *subtype_id);
+                }
             },
             ParserTypeVariant::IntegerLiteral => {
                 debug_assert!(false, "Encountered 'IntegerLiteral' symbolic type. Expected fully inferred types");
                 unreachable!();
             },
             ParserTypeVariant::Inferred => {
                 debug_assert!(false, "Encountered 'Inferred' symbolic type. Expected fully inferred types");
                 unreachable!();
+            }
+        }
+    }
+}
 enum ResolveResult {
     BuiltIn,
     PolyArg,
     Resolved((RootId, DefinitionId)),
     Unresolved((RootId, DefinitionId))
+}
 pub(crate) struct TypeTable {
     /// Lookup from AST DefinitionId to a defined type. Considering possible
     /// polymorphs is done inside the `DefinedType` struct.
     lookup: HashMap<DefinitionId, DefinedType>,
     /// Iterator over (module, definition) tuples used as workspace to make sure
     /// that each base definition of all a type's subtypes are resolved.
     iter: TypeIterator,
+}
 pub(crate) struct TypeCtx<'a> {
     symbols: &'a SymbolTable,
     heap: &'a Heap,
     modules: &'a [LexedModule]
+}
 impl TypeTable {
     /// Construct a new type table without any resolved types. Types will be
     /// resolved on-demand.
     pub(crate) fn new(ctx: &TypeCtx) -> Self {
         // Use context to guess hashmap size
         let reserve_size = ctx.heap.definitions.len();
         Self{
             lookup: HashMap::with_capacity(reserve_size),
             iter: TypeIterator::new(),
+        }
+    }
     pub(crate) fn resolve_type(&mut self, ctx: &TypeCtx, specified_type: &SpecifiedType) {
         // Make sure we're allowed to cast root_id to index into ctx.modules
         if cfg!(debug_assertions) {
             for (index, module) in ctx.modules.iter().enumerate() {
                 debug_assert_eq!(index, module.root_id.index as usize);
+            }
+        }
         // Check if we already have the definition
         let definition = self.get_definition(definition_id);
+    }
     /// This function will resolve just the basic definition of the type, it
     /// will not handle any of the monomorphized instances of the type.
     fn resolve_base_definition(&mut self, ctx: &TypeCtx, definition_id: DefinitionId) -> &DefinedType {
         // Check if we have already resolved the base definition
         if let Some(definition) = self.lookup.get(&definition_id) {
             return definition;
+        }
         let root_id = Self::find_root_id(ctx, definition_id);
         self.iter.reset(root_id, definition_id);
         while let Some((root_id, definition_id)) = self.iter.top() {
             // We have a type to resolve
             let definition = &ctx.heap[definition_id];
             let can_pop_breadcrumb = match definition {
                 Definition::Enum(definition) => self.resolve_base_enum_definition(ctx, root_id, definition),
                 Definition::Struct(definition) => self.resolve_base_struct_definition(ctx, root_id, definition),
                 Definition::Component(definition) => self.resolve_base_component_definition(ctx, root_id, definition),
                 Definition::Function(definition) => self.resolve_base_function_definition(ctx, root_id, definition),
             }?;
             // Otherwise: `ingest_resolve_result` has pushed a new breadcrumb
             // that we must follow before we can resolve the current type
             if can_pop_breadcrumb {
                 self.iter.pop();
+            }
+        }
         // We must have resolved the type
         debug_assert!(self.lookup.contains_key(&definition_id), "base type not resolved");
         self.lookup.get(&definition_id).unwrap()
+    }
     /// Resolve the basic enum definition to an entry in the type table. It will
     /// not instantiate any monomorphized instances of polymorphic enum
     /// definitions. If a subtype has to be resolved first then this function
     /// will return `false` after calling `ingest_resolve_result`.
     fn resolve_base_enum_definition(&mut self, ctx: &TypeCtx, root_id: RootId, definition: &EnumDefinition) -> Result<bool, ParseError2> {
         debug_assert!(!self.lookup.contains_key(&definition.this.upcast()), "base enum already resolved");
         // Check if the enum should be implemented as a classic enumeration or
         // a tagged union. Keep track of variant index for error messages. Make
         // sure all embedded types are resolved.
         let mut first_tag_value = None;
         let mut first_int_value = None;
         for variant in &definition.variants {
             match &variant.value {
                 EnumVariantValue::None => {},
                 EnumVariantValue::Integer(_) => if first_int_value.is_none() {
                     first_int_value = Some(variant.position);
                 },
                 EnumVariantValue::Type(variant_type_id) => {
                     if first_tag_value.is_none() {
                         first_tag_value = Some(variant.position);
+                    }
                     // Check if the embedded type needs to be resolved
                     let resolve_result = self.resolve_base_parser_type(ctx, &definition.poly_vars, root_id, *variant_type_id)?;
                     if !self.ingest_resolve_result(ctx, resolve_result) {
                         return Ok(false)
+                    }
+                }
+            }
+        }
         if first_tag_value.is_some() && first_int_value.is_some() {
             // Not illegal, but useless and probably a programmer mistake
             let module_source = &ctx.modules[root_id.index as usize].source;
             let tag_pos = first_tag_value.unwrap();
             let int_pos = first_int_value.unwrap();
             return Err(
                 ParseError2::new_error(
                     module_source, definition.position,
                     "Illegal combination of enum integer variant(s) and enum union variant(s)"
+                )
                     .with_postfixed_info(module_source, int_pos, "Assigning an integer value here")
                     .with_postfixed_info(module_source, tag_pos, "Embedding a type in a union variant here")
             );
+        }
         // Enumeration is legal
         if first_tag_value.is_some() {
             // Implement as a tagged union
             // Determine the union variants
             let mut tag_value = -1;
             let mut variants = Vec::with_capacity(definition.variants.len());
             for variant in &definition.variants {
                 tag_value += 1;
                 let parser_type = match &variant.value {
                     EnumVariantValue::None => {
                         None
                     },
                     EnumVariantValue::Type(parser_type_id) => {
                         // Type should be resolvable, we checked this above
                         Some(*parser_type_id)
                     },
                     EnumVariantValue::Integer(_) => {
                         debug_assert!(false, "Encountered `Integer` variant after asserting enum is a discriminated union");
                         unreachable!();
+                    }
                 };
                 variants.push(UnionVariant{
                     identifier: variant.identifier.clone(),
                     parser_type,
                     tag_value,
                 })
+            }
             // Ensure union names and polymorphic args do not conflict
             self.check_identifier_collision(
                 ctx, root_id, &variants, |variant| &variant.identifier, "enum variant"
             )?;
             self.check_poly_args_collision(ctx, root_id, &definition.poly_args)?;
             // Insert base definition in type table
             let definition_id = definition.this.upcast();
             self.lookup.insert(definition_id, DefinedType {
                 ast_definition: definition_id,
                 definition: DefinedTypeVariant::Union(UnionType{
                     variants,
                     tag_representation: Self::enum_tag_type(-1, tag_value),
                 }),
                 poly_args: Vec::new(),
                 is_polymorph: self.poly_args_in_use.iter().any(|v| *v),
                 is_pointerlike: false, // TODO: @cyclic_types
                 monomorphs: Vec::new()
             });
         } else {
             // Implement as a regular enum
             debug_assert!(self.poly_args_in_use.iter().all(|v| !*v));
             let mut enum_value = -1;
             let mut min_enum_value = 0;
             let mut max_enum_value = 0;
             let mut variants = Vec::with_capacity(definition.variants.len());
             for variant in &definition.variants {
                 enum_value += 1;
                 match &variant.value {
                     EnumVariantValue::None => {
                         variants.push(EnumVariant{
                             identifier: variant.identifier.clone(),
                             value: enum_value,
                         });
                     },
                     EnumVariantValue::Integer(override_value) => {
                         enum_value = *override_value;
                         variants.push(EnumVariant{
                             identifier: variant.identifier.clone(),
                             value: enum_value,
                         });
                     },
                     EnumVariantValue::Type(_) => {
                         debug_assert!(false, "Encountered `Type` variant after asserting enum is not a discriminated union");
                         unreachable!();
+                    }
+                }
                 if enum_value < min_enum_value { min_enum_value = enum_value; }
                 else if enum_value > max_enum_value { max_enum_value = enum_value; }
+            }
             // Ensure enum names and polymorphic args do not conflict
             self.check_identifier_collision(
                 ctx, root_id, &variants, |variant| &variant.identifier, "enum variant"
             )?;
             self.check_poly_args_collision(ctx, root_id, &definition.poly_vars)?;
             let definition_id = definition.this.upcast();
             self.lookup.insert(definition_id, DefinedType {
                 ast_definition: definition_id,
                 definition: DefinedTypeVariant::Enum(EnumType{
                     variants,
                     representation: Self::enum_tag_type(min_enum_value, max_enum_value)
                 }),
                 poly_args: Vec::new(),
                 is_polymorph: false,
                 is_pointerlike: false,
                 monomorphs: Vec::new()
             });
+        }
         Ok(true)
+    }
     /// Resolves the basic struct definition to an entry in the type table. It
     /// will not instantiate any monomorphized instances of polymorphic struct
     /// definitions.
     fn resolve_base_struct_definition(&mut self, ctx: &TypeCtx, root_id: RootId, definition: &StructDefinition) -> Result<bool, ParseError2> {
         debug_assert!(!self.lookup.contains_key(&definition.this.upcast()), "base struct already resolved");
         // Make sure all fields point to resolvable types
         for field_definition in &definition.fields {
             let resolve_result = self.resolve_base_parser_type(ctx, &definition.poly_vars, root_id, field_definition.parser_type)?;
             if !self.ingest_resolve_result(ctx, resolve_result)? {
                 return Ok(false)
+            }
+        }
         // All fields types are resolved, construct base type
         let mut fields = Vec::with_capacity(definition.fields.len());
         for field_definition in &definition.fields {
             fields.push(StructField{
                 identifier: field_definition.field.clone(),
                 parser_type: field_definition.parser_type,
             })
+        }
         // And make sure no conflicts exist in field names and/or polymorphic args
         self.check_identifier_collision(
             ctx, root_id, &fields, |field| &field.identifier, "struct field"
         )?;
         self.check_poly_args_collision(ctx, root_id, &definition.poly_vars)?;
         let definition_id = definition.this.upcast();
         self.lookup.insert(definition_id, DefinedType{
             ast_definition: definition_id,
             definition: DefinedTypeVariant::Struct(StructType{
                 fields,
             }),
             poly_args: Vec::new(),
             is_polymorph: false, // TODO: @polymorphism
             is_pointerlike: false, // TODO: @cyclic
             monomorphs: Vec::new(),
         });
         Ok(true)
+    }
     /// Resolves the basic function definition to an entry in the type table. It
     /// will not instantiate any monomorphized instances of polymorphic function
     /// definitions.
     fn resolve_base_function_definition(&mut self, ctx: &TypeCtx, root_id: RootId, definition: &Function) -> Result<bool, ParseError2> {
         debug_assert!(!self.lookup.contains_key(&definition.this.upcast()), "base function already resolved");
         // Check the return type
         let resolve_result = self.resolve_base_parser_type(
             ctx, &definition.poly_vars, root_id, definition.return_type
         )?;
         if !self.ingest_resolve_result(ctx, resolve_result)? {
             return Ok(false)
+        }
         // Check the argument types
         for param_id in &definition.parameters {
             let param = &ctx.heap[*param_id];
             let resolve_result = self.resolve_base_parser_type(
                 ctx, &definition.poly_vars, root_id, param.parser_type
             )?;
             if !self.ingest_resolve_result(ctx, resolve_result)? {
                 return Ok(false)
+            }
+        }
         // Construct arguments to function
         let mut arguments = Vec::with_capacity(definition.parameters.len());
         for param_id in &definition.parameters {
             let param = &ctx.heap[*param_id];
             arguments.push(FunctionArgument{
                 identifier: param.identifier.clone(),
                 parser_type: param.parser_type,
             })
+        }
         // Check conflict of argument and polyarg identifiers
         self.check_identifier_collision(
             ctx, root_id, &arguments, |arg| &arg.identifier, "function argument"
         )?;
         self.check_poly_args_collision(ctx, root_id, &definition.poly_vars)?;
         // Construct entry in type table
         let definition_id = definition.this.upcast();
         self.lookup.insert(definition_id, DefinedType{
             ast_definition: definition_id,
             definition: DefinedTypeVariant::Function(FunctionType{
                 return_type: definition.return_type,
                 arguments,
             }),
             poly_args: Vec::new(),
             is_polymorph: false, // TODO: @polymorphism
             is_pointerlike: false, // TODO: @cyclic
             monomorphs: Vec::new(),
         });
         Ok(true)
+    }
     /// Resolves the basic component definition to an entry in the type table.
     /// It will not instantiate any monomorphized instancees of polymorphic
     /// component definitions.
     fn resolve_base_component_definition(&mut self, ctx: &TypeCtx, root_id: RootId, definition: &Component) -> Result<bool, ParseError2> {
         debug_assert!(self.lookup.contains_key(&definition.this.upcast()), "base component already resolved");
         // Check argument types
         for param_id in &definition.parameters {
             let param = &ctx.heap[*param_id];
             let resolve_result = self.resolve_base_parser_type(
                 ctx, &definition.poly_vars, root_id, param.parser_type
             )?;
             if !self.ingest_resolve_result(ctx, resolve_result)? {
                 return Ok(false)
+            }
+        }
         // Construct argument types
         let mut arguments = Vec::with_capacity(definition.parameters.len());
         for param_id in &definition.parameters {
             let param = &ctx.heap[*param_id];
             arguments.push(FunctionArgument{
                 identifier: param.identifier.clone(),
                 parser_type: param.parser_type
             })
+        }
         // Check conflict of argument and polyarg identifiers
         self.check_identifier_collision(
             ctx, root_id, &arguments, |arg| &arg.identifier, "component argument"
         )?;
         self.check_poly_args_collision(ctx, root_id, &definition.poly_vars)?;
         // Construct entry in type table
         let definition_id = definition.this.upcast();
         self.lookup.insert(definition_id, DefinedType{
             ast_definition: definition_id,
             definition: DefinedTypeVariant::Component(ComponentType{
                 variant: definition.variant,
                 arguments,
             }),
             poly_args: Vec::new(),
             is_polymorph: false, // TODO: @polymorphism,
             is_pointerlike: false, // TODO: @cyclic
             monomorphs: Vec::new(),
         });
         Ok(true)
+    }
     /// Takes a ResolveResult and returns `true` if the caller can happily
     /// continue resolving its current type, or `false` if the caller must break
     /// resolving the current type and exit to the outer resolving loop. In the
     /// latter case the `result` value was `ResolveResult::Unresolved`, implying
     /// that the type must be resolved first.
     fn ingest_resolve_result(&mut self, ctx: &TypeCtx, result: ResolveResult) -> Result<bool, ParseError2> {
         match result {
             ResolveResult::BuiltIn | ResolveResult::PolyArg => Ok(true),
             ResolveResult::Resolved(_) => Ok(true),
             ResolveResult::Unresolved((root_id, definition_id)) => {
                 if self.iter.contains(root_id, definition_id) {
                     // Cyclic dependency encountered
                     // TODO: Allow this
                     let mut error = ParseError2::new_error(
                         &ctx.modules[root_id.index as usize].source, ctx.heap[definition_id].position(),
                         "Evaluating this type definition results in a cyclic type"
                     );
                     for (breadcrumb_idx, (root_id, definition_id)) in self.iter.breadcrumbs.iter().enumerate() {
                         let msg = if breadcrumb_idx == 0 {
                             "The cycle started with this definition"
                         } else {
                             "Which depends on this definition"
                         };
                         error = error.with_postfixed_info(
                             &ctx.modules[root_id.index as usize].source,
                             ctx.heap[*definition_id].position(), msg
                         );
+                    }
                     Err(error)
                 } else {
                     // Type is not yet resolved, so push IDs on iterator and
                     // continue the resolving loop
                     self.iter.push(root_id, definition_id);
                     Ok(false)
+                }
+            }
+        }
+    }
     /// Each type definition may consist of several subtypes (the type
     /// associated with a union variant, or the type associated with a struct
     /// field). This function goes through the subtype tree and determines if
     /// each reference type is resolved or not (yet).
     fn resolve_base_parser_type(&mut self, ctx: &TypeCtx, poly_vars: &Vec<Identifier>, root_id: RootId, mut parser_type_id: ParserTypeId) -> Result<ResolveResult, ParseError2> {
         use ParserTypeVariant as PTV;
         loop {
             let parser_type = &ctx.heap[parser_type_id];
             match &parser_type.variant {
                 // Builtin types. An array is a builtin as it is implemented as a
                 // couple of pointers, so we do not require the subtype to be fully
                 // resolved.
                 PTV::Array(_) |
                 PTV::Message |
                 PTV::Bool |
                 PTV::Byte |
                 PTV::Short |
                 PTV::Int |
                 PTV::Long |
                 PTV::String => {
                     return Ok(ResolveResult::BuiltIn);
                 },
                 // IntegerLiteral types and the inferred marker are not allowed in
                 // definitions of types
                 PTV::IntegerLiteral |
                 PTV::Inferred => {
                     debug_assert!(false, "Encountered illegal ParserTypeVariant within type definition");
                     unreachable!();
                 },
                 // Input/Output port
                 // TODO: @unclear Come back to this, do these need to be fully
                 //  resolved? Or do we implement these as the port identifiers?
                 PTV::Input(subtype_id) |
                 PTV::Output(subtype_id) => {
                     parser_type_id = *subtype_id;
                 },
                 PTV::Symbolic(symbolic) => {
                     // Check if the symbolic type is one of the definition's
                     // polymorphic arguments. If so then we can halt the
                     // execution
                     for poly_arg in poly_vars.iter() {
                         if poly_arg.value == symbolic.identifier.value {
                             return Ok(ResolveResult::PolyArg)
+                        }
+                    }
                     // Lookup the definition in the symbol table
                     let symbol = ctx.symbols.resolve_namespaced_symbol(root_id, &symbolic.identifier);
                     if symbol.is_none() {
                         return Err(ParseError2::new_error(
                             &ctx.modules[root_id.index as usize].source, symbolic.identifier.position,
                             "Could not resolve type"
                         ))
+                    }
                     let (symbol_value, mut ident_iter) = symbol.unwrap();
                     match symbol_value.symbol {
                         Symbol::Namespace(_) => {
                             // Reference to a namespace instead of a type
                             return if ident_iter.num_remaining() == 0 {
                                 Err(ParseError2::new_error(
                                     &ctx.modules[root_id.index as usize].source, symbolic.identifier.position,
                                     "Expected a type, got a module name"
                                 ))
                             } else {
                                 let next_identifier = ident_iter.next().unwrap();
                                 Err(ParseError2::new_error(
                                     &ctx.modules[root_id.index as usize].source, symbolic.identifier.position,
                                     &format!("Could not find symbol '{}' with this module", String::from_utf8_lossy(next_identifier))
                                 ))
+                            }
                         },
                         Symbol::Definition((root_id, definition_id)) => {
                             let definition = &ctx.heap[definition_id];
                             if ident_iter.num_remaining > 0 {
                                 // Namespaced identifier is longer than the type
                                 // we found. Return the appropriate message
                                 return if definition.is_struct() || definition.is_enum() {
                                     Err(ParseError2::new_error(
                                         &ctx.modules[root_id.index as usize].source, symbolic.identifier.position,
                                         &format!(
                                             "Unknown type '{}', did you mean to use '{}'?",
                                             String::from_utf8_lossy(&symbolic.identifier.value),
                                             String::from_utf8_lossy(&definition.identifier().value)
+                                        )
                                     ))
                                 } else {
                                     Err(ParseError2::new_error(
                                         &ctx.modules[root_id.index as usize].source, symbolic.identifier.position,
                                         "Unknown type"
                                     ))
+                                }
+                            }
                             // Found a match, make sure it is a datatype
                             if !(definition.is_struct() || definition.is_enum()) {
                                 return Err(ParseError2::new_error(
                                     &ctx.modules[root_id.index as usize].source, symbolic.identifier.position,
                                     "Embedded types must be datatypes (structs or enums)"
                                 ))
+                            }
                             // Found a struct/enum definition
                             return if self.lookup.contains_key(&definition_id) {
                                 Ok(ResolveResult::Resolved((root_id, definition_id)))
                             } else {
                                 Ok(ResolveResult::Unresolved((root_id, definition_id)))
+                            }
+                        }
+                    }
+                }
+            }
+        }
+    }
     /// Go through a list of identifiers and ensure that all identifiers have
     /// unique names
     fn check_identifier_collision<T: Sized, F: Fn(&T) -> &Identifier>(
         &self, ctx: &TypeCtx, root_id: RootId, items: &[T], getter: F, item_name: &'static str
     ) -> Result<(), ParseError2> {
         for (item_idx, item) in items.iter().enumerate() {
             let item_ident = getter(item);
             for other_item in &items[0..item_idx] {
                 let other_item_ident = getter(other_item);
                 if item_ident.value == other_item_ident {
                     let module_source = &ctx.modules[root_id.index as usize].source;
                     return Err(ParseError2::new_error(
                         module_source, item_ident.position, &format!("This {} is defined more than once", item_name)
                     ).with_postfixed_info(
                         module_source, other_item_ident.position, &format!("The other {} is defined here", item_name)
                     ));
+                }
+            }
+        }
         Ok(())
+    }
     /// Go through a list of polymorphic arguments and make sure that the
     /// arguments all have unique names, and the arguments do not conflict with
     /// any symbols defined at the module scope.
     fn check_poly_args_collision(
         &self, ctx: &TypeCtx, root_id: RootId, poly_args: &[Identifier]
     ) -> Result<(), ParseError2> {
         // Make sure polymorphic arguments are unique and none of the
         // identifiers conflict with any imported scopes
         for (arg_idx, poly_arg) in poly_args.iter().enumerate() {
             for other_poly_arg in &poly_args[..arg_idx] {
                 if poly_arg.value == other_poly_arg.value {
                     let module_source = &ctx.modules[root_id.index as usize].source;
                     return Err(ParseError2::new_error(
                         module_source, poly_arg.position,
                         "This polymorphic argument is defined more than once"
                     ).with_postfixed_info(
                         module_source, other_poly_arg.position,
                         "It conflicts with this polymorphic argument"
                     ));
+                }
+            }
             // Check if identifier conflicts with a symbol defined or imported
             // in the current module
             if let Some(symbol) = ctx.symbols.resolve_symbol(root_id, &poly_arg.value) {
                 // We have a conflict
                 let module_source = &ctx.modules[root_id.index as usize].source;
                 return Err(ParseError2::new_error(
                     module_source, poly_arg.position,
                     "This polymorphic argument conflicts with another symbol"
                 ).with_postfixed_info(
                     module_source, symbol.position,
                     "It conflicts due to this symbol"
                 ));
+            }
+        }
         // All arguments are fine
         Ok(())
+    }
     //--------------------------------------------------------------------------
     // Small utilities
     //--------------------------------------------------------------------------
     fn enum_tag_type(min_tag_value: i64, max_tag_value: i64) -> PrimitiveType {
         // TODO: @consistency tag values should be handled correctly
         debug_assert!(min_tag_value < max_tag_value);
         let abs_max_value = min_tag_value.abs().max(max_tag_value.abs());
         if abs_max_value <= u8::max_value() as i64 {
             PrimitiveType::Byte
         } else if abs_max_value <= u16::max_value() as i64 {
             PrimitiveType::Short
         } else if abs_max_value <= u32::max_value() as i64 {
             PrimitiveType::Int
         } else {
             PrimitiveType::Long
+        }
+    }
     fn find_root_id(ctx: &TypeCtx, definition_id: DefinitionId) -> RootId {
         // TODO: Keep in lookup or something
         for module in ctx.modules {
             let root_id = module.root_id;
             let root = &ctx.heap[root_id];
             for module_definition_id in root.definitions.iter() {
                 if module_definition_id == definition_id {
                     return root_id
+                }
+            }
+        }
         debug_assert!(false, "DefinitionId without corresponding RootId");
         unreachable!();
+    }
+}
@@ \ No newline at end of file @@

0 comments (0 inline, 0 general)