CSY/reowolf Changeset - 379cffa23df8 · Centrum Wiskunde & Informatica (CWI)

Changeset - 379cffa23df8

Parent rev.

Child rev.

[Not reviewed]

0 8 0

mh - 4 years ago 2021-01-15 18:21:28
contact@maxhenger.nl

WIP on type table

8 files changed with 826 insertions and 214 deletions:

language_spec.md

src/protocol/arena.rs

src/protocol/ast.rs

136

src/protocol/containers.rs

src/protocol/lexer.rs

101

src/protocol/parser/depth_visitor.rs

src/protocol/parser/symbol_table.rs

src/protocol/parser/type_table.rs

513

0 comments (0 inline, 0 general)

language_spec.md

➞

Show inline comments

 We now introduce the various identifiers that exist within the language, we make a distinction between "any identifier" and "any identifier except for the builtin ones". Because we h
 ```
 identifier-any = ALPHA | (ALPHA | DIGIT | "_")*
 keyword =
     "composite" | "primitive" |
     type-primitive | "true" | "false" | "null" |
     "struct" | "enum" |
     "if" | "else" |
     "while" | "break" | "continue" | "return" |
     "synchronous" | "assert" |
     "goto" | "skip" | "new" | "let"
-builtin = "put" | "get" | "fires" | "create" | "sizeof" | "assert"
 builtin = "put" | "get" | "fires" | "create" | "assert"
 identifier = identifier-any WITHOUT (keyword | builtin)
 // Identifier with any number of prefixed namespaces
 ns-identifier = (identifier "::")* identifier
 ```
 We then start introducing the type system. Learning from the "mistake" of C/C++ of having types like `byte` and `short` with unspecified and compiler-dependent byte-sizes (followed by everyone using `stdint.h`), we use the Rust/Zig-like `u8`, `i16`, etc. Currently we will limit the programmer to not produce integers which take up more than 64 bits. Furthermore, as one is writing network code, it would be quite neat to be able to put non-byte-aligned integers into a struct in order to directly access meaningful bits. Hence, with restrictions introduced later, we will allow for types like `i4` or `u1`. When actually retrieving them or performing computations with them we will use the next-largest byte-size to operate on them in "registers".
 **Question**: Difference between u1 and bool? Do we allow assignments between them? What about i1 and bool?
 As the language semantics are value-based, we are prevented from returning information from functions through its arguments. We may only return information through its (single) return value. If we consider the common case of having to parse a series of bytes into a meaningful struct, we cannot return both the struct and a value as a success indicator. For this reason, we introduce algebraic datatypes (or: tagged unions, or: enums) as well.

src/protocol/arena.rs

➞

Show inline comments

@@ @@ -43,24 +43,27 @@ impl<T> Arena<T> { @@
     pub fn alloc_with_id(&mut self, f: impl FnOnce(Id<T>) -> T) -> Id<T> {
         use std::convert::TryFrom;
         let id = Id {
             index: u32::try_from(self.store.len()).expect("Out of capacity!"),
             _phantom: Default::default(),
         };
         self.store.push(f(id));
         id
+    }
     pub fn iter(&self) -> impl Iterator<Item = &T> {
         self.store.iter()
+    }
     pub fn len(&self) -> usize {
         self.store.len()
+    }
+}
 impl<T> core::ops::Index<Id<T>> for Arena<T> {
     type Output = T;
     fn index(&self, id: Id<T>) -> &Self::Output {
         self.store.index(id.index as usize)
+    }
+}
 impl<T> core::ops::IndexMut<Id<T>> for Arena<T> {
     fn index_mut(&mut self, id: Id<T>) -> &mut Self::Output {
         self.store.index_mut(id.index as usize)
+    }
+}
@@ \ No newline at end of file @@

src/protocol/ast.rs

➞

Show inline comments

@@ @@ -31,25 +31,25 @@ impl ParameterId { @@
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
 pub struct LocalId(VariableId);
 impl LocalId {
     pub fn upcast(self) -> VariableId {
         self.0
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
 pub struct DefinitionId(Id<Definition>);
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct StructId(DefinitionId);
 impl StructId {
     pub fn upcast(self) -> DefinitionId{
         self.0
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
@@ @@ -70,42 +70,24 @@ impl ComponentId { @@
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct FunctionId(DefinitionId);
 impl FunctionId {
     pub fn upcast(self) -> DefinitionId {
         self.0
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct CompositeId(ComponentId);
 impl CompositeId {
     pub fn upcast(self) -> ComponentId {
         self.0
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct PrimitiveId(ComponentId);
 impl PrimitiveId {
     pub fn upcast(self) -> ComponentId {
         self.0
+    }
+}
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct StatementId(Id<Statement>);
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct BlockStatementId(StatementId);
 impl BlockStatementId {
     pub fn upcast(self) -> StatementId {
         self.0
+    }
+}
@@ @@ -397,25 +379,25 @@ impl DefinedDeclarationId { @@
 #[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)]
 pub struct ImportedDeclarationId(DeclarationId);
 impl ImportedDeclarationId {
     pub fn upcast(self) -> DeclarationId {
         self.0
+    }
+}
 #[derive(Debug, serde::Serialize, serde::Deserialize)]
 pub struct Heap {
     // Allocators
-    // #[serde[skip]] string_alloc: StringAllocator,
+    // #[serde(skip)] string_alloc: StringAllocator,
     // Root arena, contains the entry point for different modules. Each root
     // contains lists of IDs that correspond to the other arenas.
     protocol_descriptions: Arena<Root>,
     // Contents of a file, these are the elements the `Root` elements refer to
     pragmas: Arena<Pragma>,
     pub(crate) imports: Arena<Import>,
     identifiers: Arena<Identifier>,
     type_annotations: Arena<TypeAnnotation>,
     variables: Arena<Variable>,
     pub(crate) definitions: Arena<Definition>,
     statements: Arena<Statement>,
     expressions: Arena<Expression>,
@@ @@ -724,37 +706,30 @@ impl Heap { @@
         ))
+    }
     pub fn alloc_struct_definition(&mut self, f: impl FnOnce(StructId) -> StructDefinition) -> StructId {
         StructId(DefinitionId(self.definitions.alloc_with_id(|id| {
             Definition::Struct(f(StructId(DefinitionId(id))))
         })))
+    }
     pub fn alloc_enum_definition(&mut self, f: impl FnOnce(EnumId) -> EnumDefinition) -> EnumId {
         EnumId(DefinitionId(self.definitions.alloc_with_id(|id| {
             Definition::Enum(f(EnumId(DefinitionId(id))))
         })))
+    }
     pub fn alloc_composite(&mut self, f: impl FnOnce(CompositeId) -> Composite) -> CompositeId {
         CompositeId(ComponentId(DefinitionId(self.definitions.alloc_with_id(|id| {
             Definition::Component(Component::Composite(f(CompositeId(ComponentId(DefinitionId(
                 id,
             ))))))
         }))))
+    }
     pub fn alloc_primitive(&mut self, f: impl FnOnce(PrimitiveId) -> Primitive) -> PrimitiveId {
         PrimitiveId(ComponentId(DefinitionId(self.definitions.alloc_with_id(|id| {
             Definition::Component(Component::Primitive(f(PrimitiveId(ComponentId(DefinitionId(
                 id,
             ))))))
         }))))
     pub fn alloc_component(&mut self, f: impl FnOnce(ComponentId) -> Component) -> ComponentId {
         ComponentId(DefinitionId(self.definitions.alloc_with_id(|id| {
             Definition::Component(f(ComponentId(
                 DefinitionId(id),
             )))
         })))
+    }
     pub fn alloc_function(&mut self, f: impl FnOnce(FunctionId) -> Function) -> FunctionId {
         FunctionId(DefinitionId(
             self.definitions
                 .alloc_with_id(|id| Definition::Function(f(FunctionId(DefinitionId(id))))),
         ))
+    }
     pub fn alloc_pragma(&mut self, f: impl FnOnce(PragmaId) -> Pragma) -> PragmaId {
         PragmaId(self.pragmas.alloc_with_id(|id| f(PragmaId(id))))
+    }
     pub fn alloc_import(&mut self, f: impl FnOnce(ImportId) -> Import) -> ImportId {
         ImportId(self.imports.alloc_with_id(|id| f(ImportId(id))))
@@ @@ -855,38 +830,24 @@ impl Index<ComponentId> for Heap { @@
     fn index(&self, index: ComponentId) -> &Self::Output {
         &self.definitions[(index.0).0].as_component()
+    }
+}
 impl Index<FunctionId> for Heap {
     type Output = Function;
     fn index(&self, index: FunctionId) -> &Self::Output {
         &self.definitions[(index.0).0].as_function()
+    }
+}
 impl Index<CompositeId> for Heap {
     type Output = Composite;
     fn index(&self, index: CompositeId) -> &Self::Output {
         &self.definitions[((index.0).0).0].as_composite()
+    }
+}
 impl Index<PrimitiveId> for Heap {
     type Output = Primitive;
     fn index(&self, index: PrimitiveId) -> &Self::Output {
         &self.definitions[((index.0).0).0].as_primitive()
+    }
+}
 impl Index<StatementId> for Heap {
     type Output = Statement;
     fn index(&self, index: StatementId) -> &Self::Output {
         &self.statements[index.0]
+    }
+}
 impl IndexMut<StatementId> for Heap {
     fn index_mut(&mut self, index: StatementId) -> &mut Self::Output {
         &mut self.statements[index.0]
+    }
+}
@@ @@ -1190,24 +1151,34 @@ impl Root { @@
+        }
         None
+    }
     pub fn get_declaration(&self, h: &Heap, id: &Identifier) -> Option<DeclarationId> {
         for declaration_id in self.declarations.iter() {
             let declaration = &h[*declaration_id];
             if declaration.identifier().value == id.value {
                 return Some(*declaration_id);
+            }
+        }
         None
+    }
     pub fn get_declaration_namespaced(&self, h: &Heap, id: &NamespacedIdentifier) -> Option<DeclarationId> {
         for declaration_id in self.declarations.iter() {
             let declaration = &h[*declaration_id];
             // TODO: @fixme
             if declaration.identifier().value == id.value {
                 return Some(*declaration_id);
+            }
+        }
         None
+    }
+}
 impl SyntaxElement for Root {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Pragma {
     Version(PragmaVersion),
     Module(PragmaModule)
@@ @@ -1314,41 +1285,58 @@ pub struct Identifier { @@
     pub position: InputPosition,
     pub value: Vec<u8>
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct NamespacedIdentifier {
     pub position: InputPosition,
     pub num_namespaces: u8,
     pub value: Vec<u8>,
+}
 impl NamespacedIdentifier {
     fn iter(&self) -> NamespacedIdentifierIter {
+    pub(crate) fn iter(&self) -> NamespacedIdentifierIter {
         NamespacedIdentifierIter{
             value: &self.value,
             cur_offset: 0,
             num_returned: 0,
             num_total: self.num_namespaces
+        }
+    }
+}
 struct NamespacedIdentifierIter<'a> {
 impl PartialEq for NamespacedIdentifier {
     fn eq(&self, other: &Self) -> bool {
         return self.value == other.value
+    }
+}
 impl Eq for NamespacedIdentifier{}
 // TODO: Just keep ref to NamespacedIdentifier
 pub(crate) struct NamespacedIdentifierIter<'a> {
     value: &'a Vec<u8>,
     cur_offset: usize,
     num_returned: u8,
     num_total: u8,
+}
 impl<'a> NamespacedIdentifierIter<'a> {
     pub(crate) fn num_returned(&self) -> u8 {
         return self.num_returned;
+    }
     pub(crate) fn num_remaining(&self) -> u8 {
         return self.num_total - self.num_returned
+    }
+}
 impl<'a> Iterator for NamespacedIdentifierIter<'a> {
     type Item = &'a [u8];
     fn next(&mut self) -> Option<Self::Item> {
         if self.cur_offset >= self.value.len() {
             debug_assert_eq!(self.num_returned, self.num_total);
             None
         } else {
             debug_assert!(self.num_returned < self.num_total);
             let start = self.cur_offset;
             let mut end = start;
             while end < self.value.len() - 1 {
                 if self.value[end] == b':' && self.value[end + 1] == b':' {
@@ @@ -1368,39 +1356,53 @@ impl<'a> Iterator for NamespacedIdentifierIter<'a> { @@
             return Some(&self.value[start..]);
+        }
+    }
+}
 impl Display for Identifier {
     fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
         // A source identifier is in ASCII range.
         write!(f, "{}", String::from_utf8_lossy(&self.value))
+    }
+}
 type TypeData = Vec<u8>;
 #[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
 pub enum PrimitiveType {
     Input,
     Output,
     Message,
     Boolean,
     Byte,
     Short,
     Int,
     Long,
     Symbolic(TypeData),
     Symbolic(PrimitiveSymbolic)
+}
 // TODO: @cleanup, remove PartialEq implementations
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct PrimitiveSymbolic {
     // Phase 1: parser
     pub(crate) identifier: NamespacedIdentifier,
     // Phase 2: typing
     pub(crate) definition: Option<DefinitionId>
+}
 impl PartialEq for PrimitiveSymbolic {
     fn eq(&self, other: &Self) -> bool {
         self.identifier == other.identifier
+    }
+}
 impl Eq for PrimitiveSymbolic{}
 #[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
 pub struct Type {
     pub primitive: PrimitiveType,
     pub array: bool,
+}
 #[allow(dead_code)]
 impl Type {
     pub const INPUT: Type = Type { primitive: PrimitiveType::Input, array: false };
     pub const OUTPUT: Type = Type { primitive: PrimitiveType::Output, array: false };
     pub const MESSAGE: Type = Type { primitive: PrimitiveType::Message, array: false };
     pub const BOOLEAN: Type = Type { primitive: PrimitiveType::Boolean, array: false };
@@ @@ -1439,25 +1441,36 @@ impl Display for Type { @@
+            }
             PrimitiveType::Short => {
                 write!(f, "short")?;
+            }
             PrimitiveType::Int => {
                 write!(f, "int")?;
+            }
             PrimitiveType::Long => {
                 write!(f, "long")?;
+            }
             PrimitiveType::Symbolic(data) => {
                 // Type data is in ASCII range.
                 write!(f, "{}", String::from_utf8_lossy(&data))?;
                 if let Some(id) = &data.definition {
                     write!(
                         f, "Symbolic({}, id: {})",
                         String::from_utf8_lossy(&data.identifier.value),
                         id.0.index
                     )?;
                 } else {
                     write!(
                         f, "Symbolic({}, id: Unresolved)",
                         String::from_utf8_lossy(&data.identifier.value)
                     )?;
+                }
+            }
+        }
         if self.array {
             write!(f, "[]")
         } else {
             Ok(())
+        }
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct TypeAnnotation {
@@ @@ -1481,25 +1494,31 @@ pub enum Constant { @@
     Null, // message
     True,
     False,
     Character(CharacterData),
     Integer(IntegerData),
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Method {
     Get,
     Fires,
     Create,
     Symbolic(Identifier),
     Symbolic(MethodSymbolic)
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct MethodSymbolic {
     pub(crate) identifier: NamespacedIdentifier,
     pub(crate) definition: Option<DefinitionId>
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Field {
     Length,
     Symbolic(Identifier),
+}
 impl Field {
     pub fn is_length(&self) -> bool {
         match self {
             Field::Length => true,
             _ => false,
@@ @@ -1664,51 +1683,45 @@ impl Definition { @@
     pub fn as_component(&self) -> &Component {
         match self {
             Definition::Component(result) => result,
             _ => panic!("Unable to cast `Definition` to `Component`"),
+        }
+    }
     pub fn as_function(&self) -> &Function {
         match self {
             Definition::Function(result) => result,
             _ => panic!("Unable to cast `Definition` to `Function`"),
+        }
+    }
     pub fn as_composite(&self) -> &Composite {
         self.as_component().as_composite()
+    }
     pub fn as_primitive(&self) -> &Primitive {
         self.as_component().as_primitive()
+    }
     pub fn identifier(&self) -> &Identifier {
         match self {
             Definition::Struct(def) => &def.identifier,
             Definition::Enum(def) => &def.identifier,
-            Definition::Component(com) => com.identifier(),
+            Definition::Component(com) => &com.identifier,
             Definition::Function(fun) => &fun.identifier,
+        }
+    }
     pub fn parameters(&self) -> &Vec<ParameterId> {
         // TODO: Fix this
         static EMPTY_VEC: Vec<ParameterId> = Vec::new();
         match self {
-            Definition::Component(com) => com.parameters(),
+            Definition::Component(com) => &com.parameters,
             Definition::Function(fun) => &fun.parameters,
             _ => &EMPTY_VEC,
+        }
+    }
     pub fn body(&self) -> StatementId {
         // TODO: Fix this
         match self {
-            Definition::Component(com) => com.body(),
             Definition::Component(com) => com.body,
             Definition::Function(fun) => fun.body,
             _ => panic!("cannot retrieve body (for EnumDefinition or StructDefinition)")
+        }
+    }
+}
 impl SyntaxElement for Definition {
     fn position(&self) -> InputPosition {
         match self {
             Definition::Struct(def) => def.position,
             Definition::Enum(def) => def.position,
             Definition::Component(def) => def.position(),
@@ @@ -1762,105 +1775,42 @@ pub struct EnumVariantDefinition { @@
     pub value: EnumVariantValue,
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct EnumDefinition {
     pub this: EnumId,
     // Phase 1: parser
     pub position: InputPosition,
     pub identifier: Identifier,
     pub variants: Vec<EnumVariantDefinition>,
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Component {
     Composite(Composite),
     Primitive(Primitive),
+}
 impl Component {
     pub fn this(&self) -> ComponentId {
         match self {
             Component::Composite(com) => com.this.upcast(),
             Component::Primitive(prim) => prim.this.upcast(),
+        }
+    }
     pub fn as_composite(&self) -> &Composite {
         match self {
             Component::Composite(result) => result,
             _ => panic!("Unable to cast `Component` to `Composite`"),
+        }
+    }
     pub fn as_primitive(&self) -> &Primitive {
         match self {
             Component::Primitive(result) => result,
             _ => panic!("Unable to cast `Component` to `Primitive`"),
+        }
+    }
     fn identifier(&self) -> &Identifier {
         match self {
             Component::Composite(com) => &com.identifier,
             Component::Primitive(prim) => &prim.identifier,
+        }
+    }
     pub fn parameters(&self) -> &Vec<ParameterId> {
         match self {
             Component::Composite(com) => &com.parameters,
             Component::Primitive(prim) => &prim.parameters,
+        }
+    }
     pub fn body(&self) -> StatementId {
         match self {
             Component::Composite(com) => com.body,
             Component::Primitive(prim) => prim.body,
+        }
+    }
+}
 impl SyntaxElement for Component {
     fn position(&self) -> InputPosition {
         match self {
             Component::Composite(def) => def.position(),
             Component::Primitive(def) => def.position(),
+        }
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Composite {
     pub this: CompositeId,
     // Phase 1: parser
     pub position: InputPosition,
     pub identifier: Identifier,
     pub parameters: Vec<ParameterId>,
     pub body: StatementId,
+}
 impl SyntaxElement for Composite {
     fn position(&self) -> InputPosition {
         self.position
+    }
 #[derive(Debug, Clone, Copy, serde::Serialize, serde::Deserialize)]
 pub enum ComponentVariant {
     Primitive,
     Composite,
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Primitive {
     pub this: PrimitiveId,
 pub struct Component {
     pub this: ComponentId,
     // Phase 1: parser
     pub position: InputPosition,
     pub variant: ComponentVariant,
     pub identifier: Identifier,
     pub parameters: Vec<ParameterId>,
     pub body: StatementId,
+}
-impl SyntaxElement for Primitive {
+impl SyntaxElement for Component {
     fn position(&self) -> InputPosition {
         self.position
+    }
+}
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub struct Function {
     pub this: FunctionId,
     // Phase 1: parser
     pub position: InputPosition,
     pub return_type: TypeAnnotationId,
     pub identifier: Identifier,
@@ @@ -1922,26 +1872,26 @@ pub struct ImportedDeclaration { @@
 #[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
 pub enum Signature {
     Component(ComponentSignature),
     Function(FunctionSignature),
+}
 impl Signature {
     pub fn from_definition(h: &Heap, def: DefinitionId) -> Signature {
         // TODO: Fix this
         match &h[def] {
             Definition::Component(com) => Signature::Component(ComponentSignature {
                 identifier: com.identifier().clone(), // TODO: @fix
                 arity: Signature::convert_parameters(h, com.parameters()),
                 identifier: com.identifier.clone(), // TODO: @fix
                 arity: Signature::convert_parameters(h, &com.parameters),
             }),
             Definition::Function(fun) => Signature::Function(FunctionSignature {
                 return_type: h[fun.return_type].the_type.clone(),
                 identifier: fun.identifier.clone(), // TODO: @fix
                 arity: Signature::convert_parameters(h, &fun.parameters),
             }),
             _ => panic!("cannot retrieve signature (for StructDefinition or EnumDefinition)")
+        }
+    }
     fn convert_parameters(h: &Heap, params: &Vec<ParameterId>) -> Vec<Type> {
         let mut result = Vec::new();
         for &param in params.iter() {

src/protocol/containers.rs

➞

Show inline comments

 /// Containers.rs
 ///
 /// Contains specialized containers for the parser/compiler
 /// TODO: Actually implement, I really want to remove all of the identifier
 ///     allocations.
 use std::collections::LinkedList;
 const PAGE_SIZE: usize = 4096;
 struct StringPage {
     buffer: [u8; PAGE_SIZE],
     remaining: usize,
     next_page: Option<Box<StringPage>>,
+}
 impl StringPage {
     fn new() -> Self{
-        let res = Self{
         Self{
             buffer: [0; PAGE_SIZE],
             remaining: PAGE_SIZE,
             next_page: None
         };
         res
+        }
+    }
+}
 /// Custom allocator for strings that are copied and remain valid during the
 /// complete compilation phase. May perform multiple allocations but the
 /// previously allocated strings remain valid. Because we will usually allocate
 /// quite a number of these we will allocate a buffer upon construction of the
 /// StringAllocator.
 pub(crate) struct StringAllocator {
     first_page: Box<StringPage>,
     last_page: *mut StringPage,
+}
 unsafe impl Send for StringAllocator {}
 impl StringAllocator {
     pub(crate) fn new() -> StringAllocator {
         let mut page = Box::new(StringPage::new());
         let page_ptr = unsafe { page.as_mut() as *mut StringPage };
         StringAllocator{
             first_page: page,
             last_page: page_ptr,
+        }
+    }
     pub(crate) fn alloc(&mut self, data: &[u8]) -> Result<&'static str, String> {
         let data_len = data.len();
@@ @@ -81,28 +86,32 @@ impl StringAllocator { @@
 #[cfg(test)]
 mod tests {
     use super::*;
     #[test]
     fn test_alloc() {
         // Make sure pointers are somewhat correct
         let mut alloc = StringAllocator::new();
         assert!(alloc.first_page.next_page.is_none());
         assert_eq!(alloc.first_page.as_ref() as *const StringPage, alloc.last_page);
         // Insert and make page full, should not allocate another page yet
         let input = "I am a simple static string";
         let filler = " ".repeat(PAGE_SIZE - input.len());
         let ref_first = alloc.alloc(input.as_bytes()).expect("alloc first");
         let ref_filler = alloc.alloc(filler.as_bytes()).expect("alloc filler");
         assert!(alloc.first_page.next_page.is_none());
         assert_eq!(alloc.first_page.as_ref() as *const StringPage, alloc.last_page);
         let ref_second = alloc.alloc(input.as_bytes()).expect("alloc second");
         assert!(alloc.first_page.next_page.is_some());
         assert!(alloc.first_page.next_page.as_ref().unwrap().next_page.is_none());
         let last_page_ptr = alloc.first_page.next_page.as_ref().unwrap().as_ref() as *const StringPage;
         assert_eq!(last_page_ptr, alloc.last_page);
         assert_eq!(ref_first, input);
         assert_eq!(ref_filler, filler);
         assert_eq!(ref_second, input);
+    }
+}
@@ \ No newline at end of file @@

src/protocol/lexer.rs

➞

Show inline comments

 use crate::protocol::ast::*;
 use crate::protocol::inputsource::*;
 const MAX_LEVEL: usize = 128;
 const MAX_NAMESPACES: u8 = 8; // only three levels are supported at the moment
 fn is_vchar(x: Option<u8>) -> bool {
     if let Some(c) = x {
         c >= 0x21 && c <= 0x7E
     } else {
         false
+    }
+}
 fn is_wsp(x: Option<u8>) -> bool {
     if let Some(c) = x {
         c == b' ' || c == b'\t'
@@ @@ -250,83 +251,118 @@ impl Lexer<'_> { @@
                 i64::from_str_radix(&data, 10)
             };
             if let Err(_err) = parsed {
                 return Err(ParseError2::new_error(&self.source, position, "Invalid integer constant"));
+            }
             Ok(parsed.unwrap())
+        }
+    }
     // Statement keywords
+    // TODO: Clean up these functions
     fn has_statement_keyword(&self) -> bool {
         self.has_keyword(b"channel")
             || self.has_keyword(b"skip")
             || self.has_keyword(b"if")
             || self.has_keyword(b"while")
             || self.has_keyword(b"break")
             || self.has_keyword(b"continue")
             || self.has_keyword(b"synchronous")
             || self.has_keyword(b"return")
             || self.has_keyword(b"assert")
             || self.has_keyword(b"goto")
             || self.has_keyword(b"new")
             || self.has_keyword(b"put")
+            || self.has_keyword(b"put") // TODO: @fix, should be a function, even though it has sideeffects
+    }
     fn has_type_keyword(&self) -> bool {
         self.has_keyword(b"in")
             || self.has_keyword(b"out")
             || self.has_keyword(b"msg")
             || self.has_keyword(b"boolean")
             || self.has_keyword(b"byte")
             || self.has_keyword(b"short")
             || self.has_keyword(b"int")
             || self.has_keyword(b"long")
+    }
     fn has_builtin_keyword(&self) -> bool {
         self.has_keyword(b"get")
             || self.has_keyword(b"fires")
             || self.has_keyword(b"create")
             || self.has_keyword(b"length")
+    }
     fn has_reserved(&self) -> bool {
         self.has_statement_keyword()
             || self.has_type_keyword()
             || self.has_builtin_keyword()
             || self.has_keyword(b"let")
             || self.has_keyword(b"struct")
             || self.has_keyword(b"enum")
             || self.has_keyword(b"true")
             || self.has_keyword(b"false")
             || self.has_keyword(b"null")
+    }
     // Identifiers
     fn has_identifier(&self) -> bool {
         if self.has_statement_keyword() || self.has_type_keyword() || self.has_builtin_keyword() {
             return false;
+        }
         let next = self.source.next();
         is_ident_start(next)
+    }
-    fn consume_identifier(&mut self, h: &mut Heap) -> Result<Identifier, ParseError2> {
     fn consume_identifier(&mut self) -> Result<Identifier, ParseError2> {
         if self.has_statement_keyword() || self.has_type_keyword() || self.has_builtin_keyword() {
             return Err(self.error_at_pos("Expected identifier"));
+        }
         let position = self.source.pos();
         let value = self.consume_ident()?;
         Ok(Identifier{ position, value })
+    }
     fn consume_identifier_spilled(&mut self) -> Result<(), ParseError2> {
         if self.has_statement_keyword() || self.has_type_keyword() || self.has_builtin_keyword() {
             return Err(self.error_at_pos("Expected identifier"));
+        }
         self.consume_ident()?;
         Ok(())
+    }
     fn has_namespaced_identifier(&self) -> bool {
         self.has_identifier()
+    }
     fn consume_namespaced_identifier(&mut self) -> Result<NamespacedIdentifier, ParseError2> {
         if self.has_reserved() {
             return Err(self.error_at_pos("Encountered reserved keyword"));
+        }
     // Types and type annotations
         let position = self.source.pos();
         let mut ns_ident = self.consume_ident()?;
         let mut num_namespaces = 1;
         while self.has_string(b"::") {
             if num_namespaces >= MAX_NAMESPACES {
                 return Err(self.error_at_pos("Too many namespaces in identifier"));
+            }
             let new_ident = self.consume_ident()?;
             num_namespaces += 1;
+        }
         Ok(NamespacedIdentifier{
             position,
             value: ns_ident,
             num_namespaces,
         })
+    }
     // Types and type annotations
     fn consume_primitive_type(&mut self) -> Result<PrimitiveType, ParseError2> {
         if self.has_keyword(b"in") {
             self.consume_keyword(b"in")?;
             Ok(PrimitiveType::Input)
         } else if self.has_keyword(b"out") {
             self.consume_keyword(b"out")?;
             Ok(PrimitiveType::Output)
         } else if self.has_keyword(b"msg") {
             self.consume_keyword(b"msg")?;
             Ok(PrimitiveType::Message)
         } else if self.has_keyword(b"boolean") {
             self.consume_keyword(b"boolean")?;
@@ @@ -334,27 +370,32 @@ impl Lexer<'_> { @@
         } else if self.has_keyword(b"byte") {
             self.consume_keyword(b"byte")?;
             Ok(PrimitiveType::Byte)
         } else if self.has_keyword(b"short") {
             self.consume_keyword(b"short")?;
             Ok(PrimitiveType::Short)
         } else if self.has_keyword(b"int") {
             self.consume_keyword(b"int")?;
             Ok(PrimitiveType::Int)
         } else if self.has_keyword(b"long") {
             self.consume_keyword(b"long")?;
             Ok(PrimitiveType::Long)
         } else if self.has_keyword(b"let") {
             return Err(self.error_at_pos("inferred types using 'let' are reserved, but not yet implemented"));
         } else {
             let data = self.consume_ident()?;
             Ok(PrimitiveType::Symbolic(data))
             let identifier = self.consume_namespaced_identifier()?;
             Ok(PrimitiveType::Symbolic(PrimitiveSymbolic{
                 identifier,
                 definition: None
             }))
+        }
+    }
     fn has_array(&mut self) -> bool {
         let backup_pos = self.source.pos();
         let mut result = false;
         match self.consume_whitespace(false) {
             Ok(_) => result = self.has_string(b"["),
             Err(_) => {}
+        }
         self.source.seek(backup_pos);
         return result;
+    }
@@ @@ -389,25 +430,25 @@ impl Lexer<'_> { @@
+    }
     fn consume_type_annotation_spilled(&mut self) -> Result<(), ParseError2> {
         self.consume_type()?;
         Ok(())
+    }
     // Parameters
     fn consume_parameter(&mut self, h: &mut Heap) -> Result<ParameterId, ParseError2> {
         let position = self.source.pos();
         let type_annotation = self.consume_type_annotation(h)?;
         self.consume_whitespace(true)?;
-        let identifier = self.consume_identifier(h)?;
         let identifier = self.consume_identifier()?;
         let id =
             h.alloc_parameter(|this| Parameter { this, position, type_annotation, identifier });
         Ok(id)
+    }
     fn consume_parameters(
         &mut self,
         h: &mut Heap,
         params: &mut Vec<ParameterId>,
     ) -> Result<(), ParseError2> {
         self.consume_string(b"(")?;
         self.consume_whitespace(false)?;
         if !self.has_string(b")") {
@@ @@ -970,25 +1011,25 @@ impl Lexer<'_> { @@
                 self.consume_string(b"]")?;
                 self.consume_whitespace(false)?;
             } else {
                 assert!(self.has_string(b"."));
                 self.consume_string(b".")?;
                 self.consume_whitespace(false)?;
                 let subject = result;
                 let field;
                 if self.has_keyword(b"length") {
                     self.consume_keyword(b"length")?;
                     field = Field::Length;
                 } else {
-                    field = Field::Symbolic(self.consume_identifier(h)?);
                     field = Field::Symbolic(self.consume_identifier()?);
+                }
                 result = h
                     .alloc_select_expression(|this| SelectExpression {
                         this,
                         position,
                         subject,
                         field,
                     })
                     .upcast();
+            }
+        }
         Ok(result)
@@ @@ -1097,26 +1138,29 @@ impl Lexer<'_> { @@
         let position = self.source.pos();
         let method;
         if self.has_keyword(b"get") {
             self.consume_keyword(b"get")?;
             method = Method::Get;
         } else if self.has_keyword(b"fires") {
             self.consume_keyword(b"fires")?;
             method = Method::Fires;
         } else if self.has_keyword(b"create") {
             self.consume_keyword(b"create")?;
             method = Method::Create;
         } else {
             let identifier = self.consume_identifier(h)?;
             method = Method::Symbolic(identifier)
             let identifier = self.consume_namespaced_identifier()?;
             method = Method::Symbolic(MethodSymbolic{
                 identifier,
                 definition: None
             })
+        }
         self.consume_whitespace(false)?;
         let mut arguments = Vec::new();
         self.consume_string(b"(")?;
         self.consume_whitespace(false)?;
         if !self.has_string(b")") {
             while self.source.next().is_some() {
                 arguments.push(self.consume_expression(h)?);
                 self.consume_whitespace(false)?;
                 if self.has_string(b")") {
                     break;
+                }
@@ @@ -1129,25 +1173,25 @@ impl Lexer<'_> { @@
             this,
             position,
             method,
             arguments,
             declaration: None,
         }))
+    }
     fn consume_variable_expression(
         &mut self,
         h: &mut Heap,
     ) -> Result<VariableExpressionId, ParseError2> {
         let position = self.source.pos();
-        let identifier = self.consume_identifier(h)?;
         let identifier = self.consume_identifier()?;
         Ok(h.alloc_variable_expression(|this| VariableExpression {
             this,
             position,
             identifier,
             declaration: None,
         }))
+    }
     // ====================
     // Statements
     // ====================
@@ @@ -1268,30 +1312,30 @@ impl Lexer<'_> { @@
         } else {
             Ok(self.consume_memory_statement(h)?.upcast())
+        }
+    }
     fn consume_channel_statement(
         &mut self,
         h: &mut Heap,
     ) -> Result<ChannelStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"channel")?;
         self.consume_whitespace(true)?;
         let from_annotation = self.create_type_annotation_output(h)?;
-        let from_identifier = self.consume_identifier(h)?;
         let from_identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b"->")?;
         self.consume_whitespace(false)?;
         let to_annotation = self.create_type_annotation_input(h)?;
-        let to_identifier = self.consume_identifier(h)?;
         let to_identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         let from = h.alloc_local(|this| Local {
             this,
             position,
             type_annotation: from_annotation,
             identifier: from_identifier,
         });
         let to = h.alloc_local(|this| Local {
             this,
             position,
             type_annotation: to_annotation,
@@ @@ -1300,46 +1344,46 @@ impl Lexer<'_> { @@
         Ok(h.alloc_channel_statement(|this| ChannelStatement {
             this,
             position,
             from,
             to,
             next: None,
         }))
+    }
     fn consume_memory_statement(&mut self, h: &mut Heap) -> Result<MemoryStatementId, ParseError2> {
         let position = self.source.pos();
         let type_annotation = self.consume_type_annotation(h)?;
         self.consume_whitespace(true)?;
-        let identifier = self.consume_identifier(h)?;
         let identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b"=")?;
         self.consume_whitespace(false)?;
         let initial = self.consume_expression(h)?;
         let variable = h.alloc_local(|this| Local { this, position, type_annotation, identifier });
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_memory_statement(|this| MemoryStatement {
             this,
             position,
             variable,
             initial,
             next: None,
         }))
+    }
     fn consume_labeled_statement(
         &mut self,
         h: &mut Heap,
     ) -> Result<LabeledStatementId, ParseError2> {
         let position = self.source.pos();
-        let label = self.consume_identifier(h)?;
         let label = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b":")?;
         self.consume_whitespace(false)?;
         let body = self.consume_statement(h)?;
         Ok(h.alloc_labeled_statement(|this| LabeledStatement {
             this,
             position,
             label,
             body,
             in_sync: None,
         }))
+    }
@@ @@ -1380,42 +1424,42 @@ impl Lexer<'_> { @@
             test,
             body,
             next: None,
             in_sync: None,
         }))
+    }
     fn consume_break_statement(&mut self, h: &mut Heap) -> Result<BreakStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"break")?;
         self.consume_whitespace(false)?;
         let label;
         if self.has_identifier() {
-            label = Some(self.consume_identifier(h)?);
             label = Some(self.consume_identifier()?);
             self.consume_whitespace(false)?;
         } else {
             label = None;
+        }
         self.consume_string(b";")?;
         Ok(h.alloc_break_statement(|this| BreakStatement { this, position, label, target: None }))
+    }
     fn consume_continue_statement(
         &mut self,
         h: &mut Heap,
     ) -> Result<ContinueStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"continue")?;
         self.consume_whitespace(false)?;
         let label;
         if self.has_identifier() {
-            label = Some(self.consume_identifier(h)?);
             label = Some(self.consume_identifier()?);
             self.consume_whitespace(false)?;
         } else {
             label = None;
+        }
         self.consume_string(b";")?;
         Ok(h.alloc_continue_statement(|this| ContinueStatement {
             this,
             position,
             label,
             target: None,
         }))
+    }
@@ @@ -1468,25 +1512,25 @@ impl Lexer<'_> { @@
         self.consume_string(b";")?;
         Ok(h.alloc_assert_statement(|this| AssertStatement {
             this,
             position,
             expression,
             next: None,
         }))
+    }
     fn consume_goto_statement(&mut self, h: &mut Heap) -> Result<GotoStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"goto")?;
         self.consume_whitespace(false)?;
-        let label = self.consume_identifier(h)?;
         let label = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_goto_statement(|this| GotoStatement { this, position, label, target: None }))
+    }
     fn consume_new_statement(&mut self, h: &mut Heap) -> Result<NewStatementId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"new")?;
         self.consume_whitespace(false)?;
         let expression = self.consume_call_expression(h)?;
         self.consume_whitespace(false)?;
         self.consume_string(b";")?;
         Ok(h.alloc_new_statement(|this| NewStatement { this, position, expression, next: None }))
@@ @@ -1540,43 +1584,43 @@ impl Lexer<'_> { @@
             Ok(self.consume_enum_definition(h)?.upcast())
         } else if self.has_keyword(b"composite") || self.has_keyword(b"primitive") {
             Ok(self.consume_component_definition(h)?.upcast())
         } else {
             Ok(self.consume_function_definition(h)?.upcast())
+        }
+    }
     fn consume_struct_definition(&mut self, h: &mut Heap) -> Result<StructId, ParseError2> {
         // Parse "struct" keyword and its identifier
         let struct_pos = self.source.pos();
         self.consume_keyword(b"struct")?;
         self.consume_whitespace(true)?;
-        let struct_ident = self.consume_identifier(h)?;
         let struct_ident = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         // Parse struct fields
         self.consume_string(b"{")?;
         let mut next = self.source.next();
         let mut fields = Vec::new();
         while next.is_some() {
             let char = next.unwrap();
             if char == b'}' {
                 break;
+            }
             // Consume field definition
             self.consume_whitespace(false)?;
             let field_position = self.source.pos();
             let field_type = self.consume_type_annotation(h)?;
             self.consume_whitespace(true)?;
-            let field_ident = self.consume_identifier(h)?;
             let field_ident = self.consume_identifier()?;
             self.consume_whitespace(false)?;
             fields.push(StructFieldDefinition{
                 position: field_position,
                 field: field_ident,
                 the_type: field_type,
             });
             // If we have a comma, then we may or may not have another field
             // definition. Otherwise we expect the struct to be fully defined
             // and expect a closing brace
             next = self.source.next();
@@ @@ -1596,41 +1640,41 @@ impl Lexer<'_> { @@
         Ok(h.alloc_struct_definition(|this| StructDefinition{
             this,
             position: struct_pos,
             identifier: struct_ident,
             fields,
         }))
+    }
     fn consume_enum_definition(&mut self, h: &mut Heap) -> Result<EnumId, ParseError2> {
         // Parse "enum" keyword and its identifier
         let enum_pos = self.source.pos();
         self.consume_keyword(b"enum")?;
         self.consume_whitespace(true)?;
-        let enum_ident = self.consume_identifier(h)?;
         let enum_ident = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         // Parse enum variants
         self.consume_string(b"{")?;
         let mut next = self.source.next();
         let mut variants = Vec::new();
         while next.is_some() {
             let char = next.unwrap();
             if char == b'}' {
                 break;
+            }
             // Consume variant identifier
             self.consume_whitespace(false)?;
             let variant_position = self.source.pos();
-            let variant_ident = self.consume_identifier(h)?;
             let variant_ident = self.consume_identifier()?;
             self.consume_whitespace(false)?;
             // Consume variant (tag) value: may be nothing, in which case it is
             // assigned automatically, may be a constant integer, or an embedded
             // type as value, resulting in a tagged union
             next = self.source.next();
             let variant_value = if let Some(b',') = next {
                 EnumVariantValue::None
             } else if let Some(b'=') = next {
                 self.source.consume();
                 self.consume_whitespace(false)?;
                 if !self.has_integer() {
@@ @@ -1676,59 +1720,64 @@ impl Lexer<'_> { @@
         if variants.is_empty() {
             return Err(ParseError2::new_error(self.source, enum_pos, "enum definition without variants"));
+        }
         Ok(h.alloc_enum_definition(|this| EnumDefinition{
             this,
             position: enum_pos,
             identifier: enum_ident,
             variants,
         }))
+    }
     fn consume_component_definition(&mut self, h: &mut Heap) -> Result<ComponentId, ParseError2> {
         // TODO: Cleanup
         if self.has_keyword(b"composite") {
-            Ok(self.consume_composite_definition(h)?.upcast())
             Ok(self.consume_composite_definition(h)?)
         } else {
-            Ok(self.consume_primitive_definition(h)?.upcast())
             Ok(self.consume_primitive_definition(h)?)
+        }
+    }
-    fn consume_composite_definition(&mut self, h: &mut Heap) -> Result<CompositeId, ParseError2> {
+    fn consume_composite_definition(&mut self, h: &mut Heap) -> Result<ComponentId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"composite")?;
         self.consume_whitespace(true)?;
-        let identifier = self.consume_identifier(h)?;
         let identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         let mut parameters = Vec::new();
         self.consume_parameters(h, &mut parameters)?;
         self.consume_whitespace(false)?;
         let body = self.consume_block_statement(h)?;
         Ok(h.alloc_composite(|this| Composite { this, position, identifier, parameters, body }))
         Ok(h.alloc_component(|this| Component {
             this, variant: ComponentVariant::Composite, position, identifier, parameters, body
         }))
+    }
-    fn consume_primitive_definition(&mut self, h: &mut Heap) -> Result<PrimitiveId, ParseError2> {
+    fn consume_primitive_definition(&mut self, h: &mut Heap) -> Result<ComponentId, ParseError2> {
         let position = self.source.pos();
         self.consume_keyword(b"primitive")?;
         self.consume_whitespace(true)?;
-        let identifier = self.consume_identifier(h)?;
         let identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         let mut parameters = Vec::new();
         self.consume_parameters(h, &mut parameters)?;
         self.consume_whitespace(false)?;
         let body = self.consume_block_statement(h)?;
         Ok(h.alloc_primitive(|this| Primitive { this, position, identifier, parameters, body }))
         Ok(h.alloc_component(|this| Component {
             this, variant: ComponentVariant::Primitive, position, identifier, parameters, body
         }))
+    }
     fn consume_function_definition(&mut self, h: &mut Heap) -> Result<FunctionId, ParseError2> {
         let position = self.source.pos();
         let return_type = self.consume_type_annotation(h)?;
         self.consume_whitespace(true)?;
-        let identifier = self.consume_identifier(h)?;
         let identifier = self.consume_identifier()?;
         self.consume_whitespace(false)?;
         let mut parameters = Vec::new();
         self.consume_parameters(h, &mut parameters)?;
         self.consume_whitespace(false)?;
         let body = self.consume_block_statement(h)?;
         Ok(h.alloc_function(|this| Function {
             this,
             position,
             return_type,
             identifier,
             parameters,
             body,
@@ @@ -2063,44 +2112,59 @@ mod tests { @@
         struct Bar{int[] one, int[] two, Qux[] three}
         ").unwrap();
         let mut lex = Lexer::new(&mut input);
         let lexed = lex.consume_protocol_description(&mut h);
         if let Err(err) = &lexed {
             println!("{}", err);
+        }
         let lexed = lexed.unwrap();
         let root = &h[lexed];
         assert_eq!(root.definitions.len(), 2);
         let symbolic_type = |v: &PrimitiveType| -> Vec<u8> {
             if let PrimitiveType::Symbolic(v) = v {
                 v.identifier.value.clone()
             } else {
                 assert!(false);
                 unreachable!();
+            }
         };
         let foo_def = h[root.definitions[0]].as_struct();
         assert_eq!(foo_def.identifier.value, b"Foo");
         assert_eq!(foo_def.fields.len(), 3);
         assert_eq!(foo_def.fields[0].field.value, b"one");
         assert_eq!(h[foo_def.fields[0].the_type].the_type, Type::BYTE);
         assert_eq!(foo_def.fields[1].field.value, b"two");
         assert_eq!(h[foo_def.fields[1].the_type].the_type, Type::SHORT);
         assert_eq!(foo_def.fields[2].field.value, b"three");
         assert_eq!(h[foo_def.fields[2].the_type].the_type.primitive, PrimitiveType::Symbolic(Vec::from("Bar".as_bytes())));
         assert_eq!(
             symbolic_type(&h[foo_def.fields[2].the_type].the_type.primitive),
             Vec::from("Bar".as_bytes())
         );
         let bar_def = h[root.definitions[1]].as_struct();
         assert_eq!(bar_def.identifier.value, b"Bar");
         assert_eq!(bar_def.fields.len(), 3);
         assert_eq!(bar_def.fields[0].field.value, b"one");
         assert_eq!(h[bar_def.fields[0].the_type].the_type, Type::INT_ARRAY);
         assert_eq!(bar_def.fields[1].field.value, b"two");
         assert_eq!(h[bar_def.fields[1].the_type].the_type, Type::INT_ARRAY);
         assert_eq!(bar_def.fields[2].field.value, b"three");
         assert_eq!(h[bar_def.fields[2].the_type].the_type.array, true);
         assert_eq!(h[bar_def.fields[2].the_type].the_type.primitive, PrimitiveType::Symbolic(Vec::from("Qux".as_bytes())));
         assert_eq!(
             symbolic_type(&h[bar_def.fields[2].the_type].the_type.primitive),
             Vec::from("Qux".as_bytes())
         );
+    }
     #[test]
     fn test_enum_definition() {
         let mut h = Heap::new();
         let mut input = InputSource::from_string("
         enum Foo {
             A = 0,
             B = 5,
             C,
             D = 0xFF,
+        }
@@ @@ -2142,25 +2206,28 @@ mod tests { @@
                 &h[*t]
             } else {
                 assert!(false);
                 unreachable!();
+            }
         };
         assert_eq!(qux_def.identifier.value, b"Qux");
         assert_eq!(qux_def.variants.len(), 3);
         assert_eq!(qux_def.variants[0].identifier.value, b"A");
         assert_eq!(enum_type(&qux_def.variants[0].value).the_type, Type::BYTE_ARRAY);
         assert_eq!(qux_def.variants[1].identifier.value, b"B");
         assert_eq!(enum_type(&qux_def.variants[1].value).the_type.array, true);
         assert_eq!(enum_type(&qux_def.variants[1].value).the_type.primitive, PrimitiveType::Symbolic(Vec::from("Bar".as_bytes())));
         if let PrimitiveType::Symbolic(t) = &enum_type(&qux_def.variants[1].value).the_type.primitive {
             assert_eq!(t.identifier.value, Vec::from("Bar".as_bytes()));
         } else { assert!(false) }
         assert_eq!(qux_def.variants[2].identifier.value, b"C");
         assert_eq!(enum_type(&qux_def.variants[2].value).the_type, Type::BYTE);
+    }
 //     #[test]
 //     fn test_lowercase() {
 //         assert_eq!(lowercase(b'a'), b'a');
 //         assert_eq!(lowercase(b'A'), b'a');
 //         assert_eq!(lowercase(b'z'), b'z');
 //         assert_eq!(lowercase(b'Z'), b'z');
 //     }

src/protocol/parser/depth_visitor.rs

➞

Show inline comments

@@ @@ -21,28 +21,28 @@ pub(crate) trait Visitor: Sized { @@
     fn visit_symbol_definition(&mut self, h: &mut Heap, def: DefinitionId) -> VisitorResult {
         recursive_symbol_definition(self, h, def)
+    }
     fn visit_struct_definition(&mut self, h: &mut Heap, def: StructId) -> VisitorResult {
         Ok(())
+    }
     fn visit_enum_definition(&mut self, h: &mut Heap, def: EnumId) -> VisitorResult {
         Ok(())
+    }
     fn visit_component_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         recursive_component_definition(self, h, def)
+    }
-    fn visit_composite_definition(&mut self, h: &mut Heap, def: CompositeId) -> VisitorResult {
+    fn visit_composite_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         recursive_composite_definition(self, h, def)
+    }
-    fn visit_primitive_definition(&mut self, h: &mut Heap, def: PrimitiveId) -> VisitorResult {
+    fn visit_primitive_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         recursive_primitive_definition(self, h, def)
+    }
     fn visit_function_definition(&mut self, h: &mut Heap, def: FunctionId) -> VisitorResult {
         recursive_function_definition(self, h, def)
+    }
     fn visit_variable_declaration(&mut self, h: &mut Heap, decl: VariableId) -> VisitorResult {
         recursive_variable_declaration(self, h, decl)
+    }
     fn visit_parameter_declaration(&mut self, _h: &mut Heap, _decl: ParameterId) -> VisitorResult {
         Ok(())
+    }
@@ @@ -227,55 +227,56 @@ fn recursive_protocol_description<T: Visitor>( @@
+}
 fn recursive_symbol_definition<T: Visitor>(
     this: &mut T,
     h: &mut Heap,
     def: DefinitionId,
 ) -> VisitorResult {
     // We clone the definition in case it is modified
     // TODO: Fix me
     match h[def].clone() {
         Definition::Struct(def) => this.visit_struct_definition(h, def.this),
         Definition::Enum(def) => this.visit_enum_definition(h, def.this),
-        Definition::Component(cdef) => this.visit_component_definition(h, cdef.this()),
         Definition::Component(cdef) => this.visit_component_definition(h, cdef.this),
         Definition::Function(fdef) => this.visit_function_definition(h, fdef.this),
+    }
+}
 fn recursive_component_definition<T: Visitor>(
     this: &mut T,
     h: &mut Heap,
     def: ComponentId,
 ) -> VisitorResult {
     match h[def].clone() {
         Component::Composite(cdef) => this.visit_composite_definition(h, cdef.this),
         Component::Primitive(pdef) => this.visit_primitive_definition(h, pdef.this),
     let component_variant = h[def].variant;
     match component_variant {
         ComponentVariant::Primitive => this.visit_primitive_definition(h, def),
         ComponentVariant::Composite => this.visit_composite_definition(h, def),
+    }
+}
 fn recursive_composite_definition<T: Visitor>(
     this: &mut T,
     h: &mut Heap,
-    def: CompositeId,
+    def: ComponentId,
 ) -> VisitorResult {
     for &param in h[def].parameters.clone().iter() {
         recursive_parameter_as_variable(this, h, param)?;
+    }
     this.visit_statement(h, h[def].body)
+}
 fn recursive_primitive_definition<T: Visitor>(
     this: &mut T,
     h: &mut Heap,
-    def: PrimitiveId,
+    def: ComponentId,
 ) -> VisitorResult {
     for &param in h[def].parameters.clone().iter() {
         recursive_parameter_as_variable(this, h, param)?;
+    }
     this.visit_statement(h, h[def].body)
+}
 fn recursive_function_definition<T: Visitor>(
     this: &mut T,
     h: &mut Heap,
     def: FunctionId,
 ) -> VisitorResult {
@@ @@ -542,25 +543,25 @@ fn recursive_call_expression<T: Visitor>( @@
 pub(crate) struct NestedSynchronousStatements {
     illegal: bool,
+}
 impl NestedSynchronousStatements {
     pub(crate) fn new() -> Self {
         NestedSynchronousStatements { illegal: false }
+    }
+}
 impl Visitor for NestedSynchronousStatements {
-    fn visit_composite_definition(&mut self, h: &mut Heap, def: CompositeId) -> VisitorResult {
+    fn visit_composite_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         assert!(!self.illegal);
         self.illegal = true;
         recursive_composite_definition(self, h, def)?;
         self.illegal = false;
         Ok(())
+    }
     fn visit_function_definition(&mut self, h: &mut Heap, def: FunctionId) -> VisitorResult {
         assert!(!self.illegal);
         self.illegal = true;
         recursive_function_definition(self, h, def)?;
         self.illegal = false;
         Ok(())
@@ @@ -588,25 +589,25 @@ impl Visitor for NestedSynchronousStatements { @@
 pub(crate) struct ChannelStatementOccurrences {
     illegal: bool,
+}
 impl ChannelStatementOccurrences {
     pub(crate) fn new() -> Self {
         ChannelStatementOccurrences { illegal: false }
+    }
+}
 impl Visitor for ChannelStatementOccurrences {
-    fn visit_primitive_definition(&mut self, h: &mut Heap, def: PrimitiveId) -> VisitorResult {
+    fn visit_primitive_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         assert!(!self.illegal);
         self.illegal = true;
         recursive_primitive_definition(self, h, def)?;
         self.illegal = false;
         Ok(())
+    }
     fn visit_function_definition(&mut self, h: &mut Heap, def: FunctionId) -> VisitorResult {
         assert!(!self.illegal);
         self.illegal = true;
         recursive_function_definition(self, h, def)?;
         self.illegal = false;
         Ok(())
@@ @@ -686,25 +687,25 @@ impl ComponentStatementReturnNew { @@
         ComponentStatementReturnNew { illegal_new: false, illegal_return: false }
+    }
+}
 impl Visitor for ComponentStatementReturnNew {
     fn visit_component_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         assert!(!(self.illegal_new || self.illegal_return));
         self.illegal_return = true;
         recursive_component_definition(self, h, def)?;
         self.illegal_return = false;
         Ok(())
+    }
-    fn visit_primitive_definition(&mut self, h: &mut Heap, def: PrimitiveId) -> VisitorResult {
+    fn visit_primitive_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         assert!(!self.illegal_new);
         self.illegal_new = true;
         recursive_primitive_definition(self, h, def)?;
         self.illegal_new = false;
         Ok(())
+    }
     fn visit_function_definition(&mut self, h: &mut Heap, def: FunctionId) -> VisitorResult {
         assert!(!(self.illegal_new || self.illegal_return));
         self.illegal_new = true;
         recursive_function_definition(self, h, def)?;
         self.illegal_new = false;
         Ok(())
@@ @@ -840,57 +841,66 @@ impl LinkCallExpressions { @@
         LinkCallExpressions { pd: None, composite: false, new_statement: false }
+    }
     fn get_declaration(
         &self,
         h: &Heap,
         id: &Identifier,
     ) -> Result<DeclarationId, VisitorError> {
         match h[self.pd.unwrap()].get_declaration(h, &id) {
             Some(id) => Ok(id),
             None => Err((id.position, "Unresolved method".to_string())),
+        }
+    }
     fn get_declaration_namespaced(
         &self, h: &Heap, id: &NamespacedIdentifier
     ) -> Result<DeclarationId, VisitorError> {
         // TODO: @fixme
         match h[self.pd.unwrap()].get_declaration_namespaced(h, id) {
             Some(id) => Ok(id),
             None => Err((id.position, "Unresolved method".to_string()))
+        }
+    }
+}
 impl Visitor for LinkCallExpressions {
     fn visit_protocol_description(&mut self, h: &mut Heap, pd: RootId) -> VisitorResult {
         self.pd = Some(pd);
         recursive_protocol_description(self, h, pd)?;
         self.pd = None;
         Ok(())
+    }
-    fn visit_composite_definition(&mut self, h: &mut Heap, def: CompositeId) -> VisitorResult {
+    fn visit_composite_definition(&mut self, h: &mut Heap, def: ComponentId) -> VisitorResult {
         assert!(!self.composite);
         self.composite = true;
         recursive_composite_definition(self, h, def)?;
         self.composite = false;
         Ok(())
+    }
     fn visit_new_statement(&mut self, h: &mut Heap, stmt: NewStatementId) -> VisitorResult {
         assert!(self.composite);
         assert!(!self.new_statement);
         self.new_statement = true;
         recursive_new_statement(self, h, stmt)?;
         self.new_statement = false;
         Ok(())
+    }
     fn visit_call_expression(&mut self, h: &mut Heap, expr: CallExpressionId) -> VisitorResult {
         if let Method::Symbolic(id) = &h[expr].method {
             // TODO: @symbol_table
-            let decl = self.get_declaration(h, id)?;
+            let decl = self.get_declaration_namespaced(h, &id.identifier)?;
             if self.new_statement && h[decl].is_function() {
                 return Err((id.position, "Illegal call expression".to_string()));
+                return Err((id.identifier.position, "Illegal call expression".to_string()));
+            }
             if !self.new_statement && h[decl].is_component() {
                 return Err((id.position, "Illegal call expression".to_string()));
+                return Err((id.identifier.position, "Illegal call expression".to_string()));
+            }
             // Set the corresponding declaration of the call
             h[expr].declaration = Some(decl);
+        }
         // A new statement's call expression may have as arguments function calls
         let old = self.new_statement;
         self.new_statement = false;
         recursive_call_expression(self, h, expr)?;
         self.new_statement = old;
         Ok(())
+    }
+}

src/protocol/parser/symbol_table.rs

➞

Show inline comments

 // TODO: Maybe allow namespaced-aliased imports. It is currently not possible
 //  to express the following:
 //      import Module.Submodule as SubMod
 //      import SubMod::{Symbol}
 //  And it is especially not possible to express the following:
 //      import SubMod::{Symbol}
 //      import Module.Submodule as SubMod
 use crate::protocol::ast::*;
 use crate::protocol::inputsource::*;
 use std::collections::{HashMap, hash_map::Entry};
 use crate::protocol::parser::LexedModule;
 #[derive(PartialEq, Eq, Hash)]
 struct SymbolKey {
     module_id: RootId,
     symbol_name: Vec<u8>,
+}
 pub(crate) enum Symbol {
     Namespace(RootId),
     Definition((RootId, DefinitionId)),
+}
 pub(crate) struct SymbolValue {
     // Position refers to the origin of the symbol definition (i.e. the module's
     // RootId that is in the key being used to lookup this value)
     position: InputPosition,
     symbol: Symbol,
     pub(crate) position: InputPosition,
     pub(crate) symbol: Symbol,
+}
 impl SymbolValue {
     pub(crate) fn is_namespace(&self) -> bool {
         match &self.symbol {
             Symbol::Namespace(_) => true,
             _ => false
+        }
+    }
     pub(crate) fn as_namespace(&self) -> Option<RootId> {
         match &self.symbol {
             Symbol::Namespace(root_id) => Some(*root_id),
             _ => None,
+        }
+    }
     pub(crate) fn as_definition(&self) -> Option<(RootId, DefinitionId)> {
         match &self.symbol {
             Symbol::Definition((root_id, definition_id)) => Some((*root_id, *definition_id)),
             _ => None,
+        }
@@ @@ -39,28 +52,24 @@ impl SymbolValue { @@
+}
 /// `SymbolTable` is responsible for two parts of the parsing process: firstly
 /// it ensures that there are no clashing symbol definitions within each file,
 /// and secondly it will resolve all symbols within a module to their
 /// appropriate definitions (in case of enums, functions, etc.) and namespaces
 /// (currently only external modules can act as namespaces). If a symbol clashes
 /// or if a symbol cannot be resolved this will be an error.
 ///
 /// Within the compilation process the symbol table is responsible for resolving
 /// namespaced identifiers (e.g. Module::Enum::EnumVariant) to the appropriate
 /// definition (i.e. not namespaces; as the language has no way to use
 /// namespaces except for using them in namespaced identifiers).
 // TODO: Maybe allow namespaced-aliased imports. It is currently not possible
 //  to express the following:
 //  import Module.Submodule as ModSub
 //  import SubMod::{Symbol}
 pub(crate) struct SymbolTable {
     // Lookup from module name (not any aliases) to the root id
     module_lookup: HashMap<Vec<u8>, RootId>,
     // Lookup from within a module, to a particular imported (potentially
     // aliased) or defined symbol. Basically speaking: if the source code of a
     // module contains correctly imported/defined symbols, then this lookup
     // will always return the corresponding definition
     symbol_lookup: HashMap<SymbolKey, SymbolValue>,
+}
 impl SymbolTable {
     pub(crate) fn new(heap: &Heap, modules: &[LexedModule]) -> Result<Self, ParseError2> {
@@ @@ -274,34 +283,101 @@ impl SymbolTable { @@
     /// with a single non-namespaced identifier
     pub(crate) fn resolve_symbol(&self, within_module_id: RootId, identifier: &Vec<u8>) -> Option<&SymbolValue> {
         self.symbol_lookup.get(&SymbolKey{ module_id: within_module_id, symbol_name: identifier.clone() })
+    }
     /// Resolves a namespaced symbol. It will try to go as far as possible in
     /// actually finding a definition or a namespace. So a namespace might be
     /// resolved, after it which it finds an actual definition. It may be that
     /// the namespaced identifier has more elements that should be checked
     /// (i.e. an enum variant, or simply an erroneous instance of too many
     /// chained identifiers). This function will return None if nothing could be
     /// resolved at all.
     pub(crate) fn resolve_namespaced_symbol(&self, _within_module_id: RootId) {
         todo!("implement")
     pub(crate) fn resolve_namespaced_symbol<'t, 'i>(
         &'t self, root_module_id: RootId, identifier: &'i NamespacedIdentifier
     ) -> Option<(&SymbolValue, NamespacedIdentifierIter<'i>)> {
         let mut iter = identifier.iter();
         let mut symbol: Option<&SymbolValue> = None;
         let mut within_module_id = root_module_id;
         while let Some(partial) = iter.next() {
             // Lookup the symbol within the currently iterated upon module
             let lookup_key = SymbolKey{ module_id: within_module_id, symbol_name: Vec::from(partial) };
             let new_symbol = self.symbol_lookup.get(&lookup_key);
             match new_symbol {
                 None => {
                     // Can't find anything
                     break;
                 },
                 Some(new_symbol) => {
                     // Found something, but if we already moved to another
                     // module then we don't want to keep jumping across modules,
                     // we're only interested in symbols defined within that
                     // module.
                     match &new_symbol.symbol {
                         Symbol::Namespace(new_root_id) => {
                             if root_module_id != within_module_id {
                                 // Don't jump from module to module, keep the
                                 // old symbol (which must be a Namespace) and
                                 // break
                                 debug_assert!(symbol.is_some());
                                 debug_assert!(symbol.unwrap().is_namespace());
                                 debug_assert!(iter.num_returned() > 1);
                                 // For handling this error, we need to revert
                                 // the iterator by one
                                 let to_skip = iter.num_returned() - 1;
                                 iter = identifier.iter();
                                 for _ in 0..to_skip { iter.next(); }
                                 break;
+                            }
                             within_module_id = *new_root_id;
                             symbol = Some(new_symbol);
                         },
                         Symbol::Definition((definition_root_id, _)) => {
                             // Found a definition, but if we already jumped
                             // modules, then this must be defined within that
                             // module.
                             if root_module_id != within_module_id && within_module_id != *definition_root_id {
                                 // This is an imported definition within the module
                                 // TODO: Maybe factor out? Dunno...
                                 debug_assert!(symbol.is_some());
                                 debug_assert!(symbol.unwrap().is_namespace());
                                 debug_assert!(iter.num_returned() > 1);
                                 let to_skip = iter.num_returned() - 1;
                                 iter = identifier.iter();
                                 for _ in 0..to_skip { iter.next(); }
                                 break;
+                            }
                             symbol = Some(new_symbol);
                             break;
+                        }
+                    }
+                }
+            }
+        }
         match symbol {
             None => None,
             Some(symbol) => Some((symbol, iter))
+        }
+    }
     /// Attempts to add a namespace symbol. Returns `Ok` if the symbol was
     /// inserted. If the symbol already exists then `Err` will be returned
     /// together with the previous definition's source position (in the origin
     /// module's source file).
     // Note: I would love to return a reference to the value, but Rust is
     // preventing me from doing so...
+    // preventing me from doing so... That, or I'm not smart enough...
     fn add_namespace_symbol(
         &mut self, origin_module_id: RootId, origin_position: InputPosition, symbol_name: &Vec<u8>, target_module_id: RootId
     ) -> Result<(), InputPosition> {
         let key = SymbolKey{
             module_id: origin_module_id,
             symbol_name: symbol_name.clone()
         };
         match self.symbol_lookup.entry(key) {
             Entry::Occupied(o) => Err(o.get().position),
             Entry::Vacant(v) => {
                 v.insert(SymbolValue{
                     position: origin_position,

src/protocol/parser/type_table.rs

➞

Show inline comments

 // TODO: @fix PrimitiveType for enums/unions
 use crate::protocol::ast::*;
 use crate::protocol::parser::symbol_table::*;
 use crate::protocol::parser::symbol_table::{SymbolTable, Symbol};
 use crate::protocol::inputsource::*;
 use crate::protocol::parser::*;
 use std::collections::HashMap;
 enum DefinedType {
+pub enum DefinedType {
     Enum(EnumType),
     Union(UnionType),
     Struct(StructType),
     Function(FunctionType),
     Component(ComponentType)
+}
 // TODO: Also support maximum u64 value
 struct EnumVariant {
+pub struct EnumVariant {
     identifier: Identifier,
     value: i64,
+}
 /// `EnumType` is the classical C/C++ enum type. It has various variants with
 /// an assigned integer value. The integer values may be user-defined,
 /// compiler-defined, or a mix of the two. If a user assigns the same enum
 /// value multiple times, we assume the user is an expert and we consider both
 /// variants to be equal to one another.
 struct EnumType {
     definition: DefinitionId,
 pub struct EnumType {
     variants: Vec<EnumVariant>,
     representation: PrimitiveType,
+}
 struct UnionVariant {
+pub struct UnionVariant {
     identifier: Identifier,
-    embedded_type: Option<DefinitionId>,
+    embedded_type: Option<TypeAnnotationId>,
     tag_value: i64,
+}
 struct UnionType {
     definition: DefinitionId,
     variants: Vec<EnumVariant>,
 pub struct UnionType {
     variants: Vec<UnionVariant>,
     tag_representation: PrimitiveType
+}
-struct StructMemberType {
+pub struct StructField {
     identifier: Identifier,
     field_type: TypeAnnotationId,
+}
 pub struct StructType {
     fields: Vec<StructField>,
+}
 pub struct FunctionArgument {
     identifier: Identifier,
     argument_type: TypeAnnotationId,
+}
 struct StructType {
 pub struct FunctionType {
     return_type: Type,
     arguments: Vec<FunctionArgument>
+}
 pub struct ComponentType {
     variant: ComponentVariant,
     arguments: Vec<FunctionArgument>
+}
 pub struct TypeTable {
     lookup: HashMap<DefinitionId, DefinedType>,
+}
 enum LookupResult {
     BuiltIn,
     Resolved((RootId, DefinitionId)),
     Unresolved((RootId, DefinitionId)),
     Error((InputPosition, String)),
+}
 struct FunctionType {
 /// `TypeTable` is responsible for walking the entire lexed AST and laying out
 /// the various user-defined types in terms of bytes and offsets. This process
 /// may be pseudo-recursive (meaning: it is implemented in a recursive fashion,
 /// but not in the recursive-function-call kind of way) in case a type depends
 /// on another type to be resolved.
 /// TODO: Distinction between resolved types and unresolved types (regarding the
 ///     mixed use of BuiltIns and resolved types) is a bit yucky at the moment,
 ///     will have to come up with something nice in the future.
 /// TODO: Need to factor out the repeated lookup in some kind of state machine.
 impl TypeTable {
     fn new(
         symbols: &SymbolTable, heap: &Heap, modules: &[LexedModule]
     ) -> Result<TypeTable, ParseError2> {
         if cfg!(debug_assertions) {
             for (index, module) in modules.iter().enumerate() {
                 debug_assert_eq!(index, module.root_id.0.index as usize)
+            }
+        }
         // Estimate total number of definitions we will encounter
         let num_definitions = heap.definitions.len();
         let mut table = TypeTable{
             lookup: HashMap::with_capacity(num_definitions),
         };
         // Perform the breadcrumb-based type parsing. For now we do not allow
         // cyclic types.
         // TODO: Allow cyclic types. However, we want to have an implementation
         //  that is somewhat efficient: if a type is cyclic, then we have to
         //  insert a pointer somewhere. However, we don't want to insert them
         //  everywhere, for each possible type. Without any further context the
         //  decision to place a pointer somewhere is "random", we need to know
         //  how the type is used to have an informed opinion on where to place
         //  the pointer.
         enum Breadcrumb {
             Linear((usize, usize)),
             Jumping((RootId, DefinitionId))
+        }
         let mut module_index = 0;
         let mut definition_index = 0;
         let mut breadcrumbs = Vec::with_capacity(32); // if a user exceeds this, the user sucks at programming
         while module_index < modules.len() {
             // Go to next module if needed
+            {
                 let root = &heap[modules[module_index].root_id];
                 if definition_index >= root.definitions.len() {
                     module_index += 1;
                     definition_index = 0;
                     continue;
+                }
+            }
             // Construct breadcrumbs in case we need to follow some types around
             debug_assert!(breadcrumbs.is_empty());
             breadcrumbs.push(Breadcrumb::Linear((module_index, definition_index)));
             'resolve_loop: while !breadcrumbs.is_empty() {
                 // Retrieve module, the module's root and the definition
                 let (module, root, definition_id) = match breadcrumbs.last().unwrap() {
                     Breadcrumb::Linear((module_index, definition_index)) => {
                         let module = &modules[*module_index];
                         let root = &heap[module.root_id];
                         let definition_id = root.definitions[*definition_index];
                         (module, root, definition_id)
                     },
                     Breadcrumb::Jumping((root_id, definition_id)) => {
                         let module = &modules[root_id.0.index as usize];
                         debug_assert_eq!(module.root_id, *root_id);
                         let root = &heap[*root_id];
                         (module, root, *definition_id)
+                    }
                 };
                 let definition = &heap[definition_id];
                 // Because we might have chased around to this particular
                 // definition before, we check if we haven't resolved the type
                 // already.
                 if table.lookup.contains_key(&definition_id) {
                     breadcrumbs.pop();
                     continue;
+                }
                 match definition {
                     Definition::Enum(definition) => {
                         // Check the definition to see if we're dealing with an
                         // enum or a union. If we find any union variants then
                         // we immediately check if the type is already resolved.
                         let mut has_tag_values = None;
                         let mut has_int_values = None;
                         for variant in &definition.variants {
                             match &variant.value {
                                 EnumVariantValue::None => {},
                                 EnumVariantValue::Integer(_) => {
                                     if has_int_values.is_none() { has_int_values = Some(variant.position); }
                                  },
                                 EnumVariantValue::Type(variant_type) => {
                                     if has_tag_values.is_none() { has_tag_values = Some(variant.position); }
                                     let variant_type = &heap[*variant_type];
                                     match lookup_type_definition(
                                         heap, &table, symbols, module.root_id,
                                         &variant_type.the_type.primitive
                                     ) {
                                         LookupResult::BuiltIn | LookupResult::Resolved(_) => {},
                                         LookupResult::Unresolved(root_and_definition_id) => {
                                             breadcrumbs.push(Breadcrumb::Jumping(root_and_definition_id));
                                             continue 'resolve_loop;
                                         },
                                         LookupResult::Error((position, message)) => {
                                             return Err(ParseError2::new_error(&module.source, position, &message));
+                                        }
+                                    }
                                 },
+                            }
+                        }
                         if has_tag_values.is_some() && has_int_values.is_some() {
                             // Not entirely illegal, but probably not desired
                             let tag_pos = has_tag_values.unwrap();
                             let int_pos = has_int_values.unwrap();
                             return Err(
                                 ParseError2::new_error(&module.source, definition.position, "Illegal combination of enum integer variant(s) and enum union variant(s)")
                                 .with_postfixed_info(&module.source, int_pos, "Explicitly assigning an integer value here")
                                 .with_postfixed_info(&module.source, tag_pos, "Explicitly declaring a union variant here")
+                            )
+                        }
                         // If here, then the definition is a valid discriminated
                         // union with all of its types resolved, or a valid
                         // enum.
                         // Decide whether to implement as enum or as union
                         let is_union = has_tag_values.is_some();
                         if is_union {
                             // Implement as discriminated union. Because we
                             // checked the availability of types above, we are
                             // safe to lookup type definitions
                             let mut tag_value = -1;
                             let mut variants = Vec::with_capacity(definition.variants.len());
                             for variant in &definition.variants {
                                 tag_value += 1;
                                 let embedded_type = match &variant.value {
                                     EnumVariantValue::None => {
                                         None
                                     },
                                     EnumVariantValue::Type(type_annotation_id) => {
                                         // Type should be resolvable, we checked this above
                                         let type_annotation = &heap[*type_annotation_id];
                                         // TODO: Remove the assert once I'm clear on how to layout "the types" of types
                                         if cfg!(debug_assertions) {
                                             ensure_type_definition(heap, &table, symbols, module.root_id, &type_annotation.the_type.primitive);
+                                        }
                                         Some(*type_annotation_id)
                                     },
                                     EnumVariantValue::Integer(_) => {
                                         debug_assert!(false, "Encountered `Integer` variant after asserting enum is a discriminated union");
                                         unreachable!();
+                                    }
                                 };
                                 variants.push(UnionVariant{
                                     identifier: variant.identifier.clone(),
                                     embedded_type,
                                     tag_value,
                                 })
+                            }
                             table.add_definition(definition_id, DefinedType::Union(UnionType{
                                 variants,
                                 tag_representation: enum_representation(tag_value)
                             }));
                         } else {
                             // Implement as regular enum
                             let mut enum_value = -1; // TODO: allow u64 max size
                             let mut variants = Vec::with_capacity(definition.variants.len());
                             for variant in &definition.variants {
                                 enum_value += 1;
                                 match &variant.value {
                                     EnumVariantValue::None => {
                                         variants.push(EnumVariant{
                                             identifier: variant.identifier.clone(),
                                             value: enum_value,
                                         });
                                     },
                                     EnumVariantValue::Integer(override_value) => {
                                         enum_value = *override_value;
                                         variants.push(EnumVariant{
                                             identifier: variant.identifier.clone(),
                                             value: enum_value,
                                         });
                                     },
                                     EnumVariantValue::Type(_) => {
                                         debug_assert!(false, "Encountered `Type` variant after asserting enum is not a discriminated union");
                                         unreachable!();
+                                    }
+                                }
+                            }
                             table.add_definition(definition_id, DefinedType::Enum(EnumType{
                                 variants,
                                 representation: enum_representation(enum_value),
                             }));
+                        }
                     },
                     Definition::Struct(definition) => {
                         // Before we start allocating fields, make sure we can
                         // actually resolve all of the field types
                         for field_definition in &definition.fields {
                             let type_definition = &heap[field_definition.the_type];
                             match lookup_type_definition(
                                 heap, &table, symbols, module.root_id,
                                 &type_definition.the_type.primitive
                             ) {
                                 LookupResult::BuiltIn | LookupResult::Resolved(_) => {},
                                 LookupResult::Unresolved(root_and_definition_id) => {
                                     breadcrumbs.push(Breadcrumb::Jumping(root_and_definition_id));
                                     continue 'resolve_loop;
                                 },
                                 LookupResult::Error((position, message)) => {
                                     return Err(ParseError2::new_error(&module.source, position, &message));
+                                }
+                            }
+                        }
                         // We can resolve everything
                         let mut fields = Vec::with_capacity(definition.fields.len());
                         for field_definition in &definition.fields {
                             let type_annotation = &heap[field_definition.the_type];
                             if cfg!(debug_assertions) {
                                 ensure_type_definition(heap, &table, symbols, module.root_id, &type_annotation.the_type.primitive);
+                            }
                             fields.push(StructField{
                                 identifier: field_definition.field.clone(),
                                 field_type: field_definition.the_type
                             });
+                        }
                         table.add_definition(definition_id, DefinedType::Struct(StructType{
                             fields,
                         }));
                     },
                     Definition::Component(definition) => {
                         // As always, ensure all parameter types are resolved
                         for parameter_id in &definition.parameters {
                             let parameter = &heap[*parameter_id];
                             let type_definition = &heap[parameter.type_annotation];
                             match lookup_type_definition(
                                 heap, &table, symbols, module.root_id,
                                 &type_definition.the_type.primitive
                             ) {
                                 LookupResult::BuiltIn | LookupResult::Resolved(_) => {},
                                 LookupResult::Unresolved(root_and_definition_id) => {
                                     breadcrumbs.push(Breadcrumb::Jumping(root_and_definition_id));
                                     continue 'resolve_loop;
                                 },
                                 LookupResult::Error((position, message)) => {
                                     return Err(ParseError2::new_error(&module.source, position, &message));
+                                }
+                            }
+                        }
                         // We can resolve everything
                         let mut parameters = Vec::with_capacity(definition.parameters.len());
                         for parameter_id in &definition.parameters {
                             let parameter = &heap[*parameter_id];
                             let type_definition = &heap[parameter.type_annotation];
                             if cfg!(debug_assertions) {
                                 ensure_type_definition(heap, &table, symbols, module.root_id, &type_definition.the_type.primitive);
+                            }
                             parameters.push(FunctionArgument{
                                 identifier: parameter.identifier.clone(),
                                 argument_type: parameter.type_annotation,
                             });
+                        }
                         table.add_definition(definition_id, DefinedType::Component(ComponentType{
                             variant: definition.variant,
                             arguments: parameters, // Arguments, parameters, tomayto, tomahto
                         }));
                     },
                     Definition::Function(definition) => {
                         // TODO: Onto the last one!
                     },
+                }
                 // If here, then we layed out the current type definition under
                 // investigation, so:
                 debug_assert!(!breadcrumbs.is_empty());
                 breadcrumbs.pop();
+            }
             // Go to next definition
             definition_index += 1;
+        }
         debug_assert_eq!(
             num_definitions, table.lookup.len(),
             "expected {} (reserved) definitions in table, got {}",
             num_definitions, table.lookup.len()
         );
         Ok(table)
+    }
     pub(crate) fn get_definition(&self, definition_id: &DefinitionId) -> Option<&DefinedType> {
         self.lookup.get(definition_id)
+    }
     fn add_definition(&mut self, definition_id: DefinitionId, definition: DefinedType) {
         debug_assert!(!self.lookup.contains_key(&definition_id), "already added definition");
         self.lookup.insert(definition_id, definition);
+    }
+}
 struct ComponentType {
 /// Attempts to lookup a type definition using a namespaced identifier. We have
 /// three success cases: the first is simply that the type is a `BuiltIn` type.
 /// In the two other cases both we find the definition of the type in the symbol
 /// table, but in one case it is already `Resolved` in the type table, and in
 /// the other case it is `Unresolved`. In this last case the type has to be
 /// resolved before we're able to use it in the `TypeTable` construction
 /// algorithm.
 /// In the `Error` case something goes wrong with resolving the type. The error
 /// message aims to be as helpful as possible to the user.
 /// The caller should ensure that the `module_root` is where the `identifier`
 /// lives.
 fn lookup_type_definition(
     heap: &Heap, types: &TypeTable, symbols: &SymbolTable,
     module_root: RootId, type_to_resolve: &PrimitiveType
 ) -> LookupResult {
     if let PrimitiveType::Symbolic(type_to_resolve) = type_to_resolve {
         let identifier = &type_to_resolve.identifier;
         match symbols.resolve_namespaced_symbol(module_root, identifier) {
             None => {
                 // Failed to find anything at all
                 LookupResult::Error((identifier.position, String::from("Unknown type")))
             },
             Some((symbol, mut identifier_iter)) => {
                 match symbol.symbol {
                     Symbol::Namespace(_root_id) => {
                         // Reference to a namespace, which is not a type. However,
                         // the error message depends on whether we have identifiers
                         // remaining or not
                         if identifier_iter.num_remaining() == 0 {
                             LookupResult::Error((
                                 identifier.position,
                                 String::from("Expected a type, got a module name")
                             ))
                         } else {
                             let next_identifier = identifier_iter.next().unwrap();
                             LookupResult::Error((
                                 identifier.position,
                                 format!("Cannot find symbol '{}' in this module", String::from_utf8_lossy(next_identifier))
                             ))
+                        }
                     },
                     Symbol::Definition((definition_root_id, definition_id)) => {
                         // Got a definition, but we may also have more identifier's
                         // remaining in the identifier iterator
                         let definition = &heap[definition_id];
                         if identifier_iter.num_remaining() == 0 {
                             // See if the type is resolved, and make sure it is
                             // a non-function, non-component type. Ofcourse
                             // these are valid types, but we cannot (yet?) use
                             // them as function arguments, struct fields or enum
                             // variants
                             match definition {
                                 Definition::Component(definition) => {
                                     return LookupResult::Error((
                                         identifier.position,
                                         format!(
                                             "Cannot use the component '{}' as an embedded type",
                                             String::from_utf8_lossy(&definition.identifier.value)
+                                        )
                                     ));
                                 },
                                 Definition::Function(definition) => {
                                     return LookupResult::Error((
                                         identifier.position,
                                         format!(
                                             "Cannot use the function '{}' as an embedded type",
                                             String::from_utf8_lossy(&definition.identifier.value)
+                                        )
                                     ));
                                 },
                                 Definition::Enum(_) | Definition::Struct(_) => {}
+                            }
                             return if types.lookup.contains_key(&definition_id) {
                                 LookupResult::Resolved((definition_root_id, definition_id))
                             } else {
                                 LookupResult::Unresolved((definition_root_id, definition_id))
+                            }
                         } else if identifier_iter.num_remaining() == 1 {
                             // This is always invalid, but if the type is an
                             // enumeration or a union then we want to return a
                             // different error message.
                             if definition.is_enum() {
                                 let last_identifier = identifier_iter.next().unwrap();
                                 return LookupResult::Error((
                                     identifier.position,
                                     format!(
                                         "Expected a type, but got a (possible) enum variant '{}'. Only the enum '{}' itself can be used as a type",
                                         String::from_utf8_lossy(last_identifier),
                                         String::from_utf8_lossy(&definition.identifier().value)
+                                    )
                                 ));
+                            }
+                        }
                         // Too much identifiers (>1) for an enumeration, or not an
                         // enumeration and we had more identifiers remaining
                         LookupResult::Error((
                             identifier.position,
                             format!(
                                 "Unknown type '{}', did you mean to use '{}'?",
                                 String::from_utf8_lossy(&identifier.value),
                                 String::from_utf8_lossy(&definition.identifier().value)
+                            )
                         ))
+                    }
+                }
+            }
+        }
     } else {
         LookupResult::BuiltIn
+    }
+}
 struct TypeTable {
     lookup: HashMap<DefinitionId, DefinedType>,
 /// Debugging function to ensure a type is resolved (calling
 /// `lookup_type_definition` and ensuring its return value is `BuiltIn` or
 /// `Resolved`
 #[cfg(debug_assertions)]
 fn ensure_type_definition(
     heap: &Heap, types: &TypeTable, symbols: &SymbolTable,
     module_root: RootId, type_to_resolve: &PrimitiveType
 ) {
     match lookup_type_definition(heap, types, symbols, module_root, type_to_resolve) {
         LookupResult::BuiltIn | LookupResult::Resolved(_) => {},
         LookupResult::Unresolved((_, definition_id)) => {
             assert!(
                 false,
                 "Expected that type definition for {} was resolved by the type table, but it wasn't",
                 String::from_utf8_lossy(&heap[definition_id].identifier().value)
+            )
         },
         LookupResult::Error((_, error)) => {
             let message = if let PrimitiveType::Symbolic(symbolic) = type_to_resolve {
                 format!(
                     "Expected that type definition for {} was resolved by the type table, but it returned: {}",
                     String::from_utf8_lossy(&symbolic.identifier.value), &error
+                )
             } else {
                 format!(
                     "Expected (non-symbolic!?) type definition to be resolved, but it returned: {}",
                     &error
+                )
             };
             assert!(false, "{}", message)
+        }
+    }
+}
 /// Determines an enumeration's integer representation type, or a union's tag
 /// type, using the maximum value of the tag. The returned type is always a
 /// builtin type.
 /// TODO: Fix for maximum u64 value
 fn enum_representation(max_tag_value: i64) -> PrimitiveType {
     if max_tag_value <= u8::max_value() as i64 {
         PrimitiveType::Byte
     } else if max_tag_value <= u16::max_value() as i64 {
         PrimitiveType::Short
     } else if max_tag_value <= u32::max_value() as i64 {
         PrimitiveType::Int
     } else {
         PrimitiveType::Long
+    }
+}
@@ \ No newline at end of file @@

0 comments (0 inline, 0 general)