Commit graph

238 commits

Author SHA1 Message Date
Yann Lanthony
20be401110 [core] CompatibilityNode: use original serialized inputs if not in graph
renamed 'actualInputs' property to 'inputs', which returns the serialized inputs depending on whether the node belongs to a graph or not
2018-07-24 17:28:14 +02:00
Yann Lanthony
6bb70895f3 [core] remove ABCMeta metaclass for Python 2 compatibility
mixing ABCMeta and QObject inheritance is problematic when using Python 2
2018-07-24 15:06:15 +02:00
Fabien Castan
9e06fd9661 [core] generic functions to create "writing" filepath and rename the file
* only try to remove the file on windows
* only except for specific exception
* use the same functions to create the statistics file
2018-07-24 14:32:24 +02:00
Fabien Castan
c6531bcc7b [core] multiple trials to replace the status file to avoid file conflicts on windows
On Windows, attempting to remove a file that is in use causes an exception to be raised. So we may need multiple trials.
2018-07-21 14:50:07 +02:00
Yann Lanthony
dd3d9cd54b [core][ui] add upgradeAllNodes methods
+ check if node is upgradable in UpgradeNodeCommand
2018-07-20 17:55:30 +02:00
Yann Lanthony
8679fd8bf7 [core] maintain a model with CompatibilityNodes on topology change 2018-07-20 17:52:30 +02:00
Yann Lanthony
3417f58335 [graph] add GraphModification inside upgradeNode method 2018-07-20 15:35:55 +02:00
Yann Lanthony
f5820b3eb1 [graph] expose fileReleaseVersion as property
+ clear header in Graph.clear method
2018-07-20 15:30:27 +02:00
Yann Lanthony
ae716602ab [core] update nodes topological data after nodes status update
evaluate nodes computability requires up-to-date status
2018-07-19 16:08:00 +02:00
Yann Lanthony
5897dc13b6 [core] verify chunk hosts a subprocess before killing it
a chunk can be marked as "RUNNING" without hosting a subprocess, if a node is duplicated in the graph
2018-07-19 13:50:45 +02:00
Yann Lanthony
f5d11a8a7a [node] CompatibilityNode: expose additional properties
* add 'issueDetails' to get a comprehensive text about the issue
2018-07-19 13:50:45 +02:00
Yann Lanthony
30cd8001fd [core][graph] update nodes computability on topology change
CompatibilityNodes introduce a new notion of computability per node.
An uncomputed (and therefore uncomputable) CompatibilityNode blocks the computation of all its successor.

* update nodes computability on topology change (in addition to min/max depth)
* evaluate leaves computability to determine if the whole graph can be processed
2018-07-19 13:50:45 +02:00
Yann Lanthony
89dd55f43b [core] dfsToProcess: stop branch visit when discovering a computed node 2018-07-19 13:50:45 +02:00
Yann Lanthony
ab4e82aa88 [core] add GraphVisitMessage exceptions mecanism
allow to stop branch or graph visit by throwing specific exceptions in visitor callbacks
2018-07-19 13:50:45 +02:00
Yann Lanthony
52d1d1fb89 [core] node_factory: del old 'attributes' key from serialized node dict 2018-07-19 13:50:45 +02:00
Yann Lanthony
2952e11691 [core] move duplicateNode methods to core.graph
handle this low-level operation engine side

* fix ListAttribute children links duplication
* handle CompatibilityNode duplication
* move corresponding unit test in test_graph.py
* [ui] add DuplicateNodeCommand
2018-07-19 13:50:45 +02:00
Yann Lanthony
1af3a16d81 [core] CompatibilityNode: recursively build List/GroupAttribute descriptions 2018-07-19 13:50:45 +02:00
Yann Lanthony
1d309136ec [core][node] serialization: rename 'attributes' to 'inputs' 2018-07-19 13:50:45 +02:00
Yann Lanthony
33eb7f3a7f [core] introduce CompatibilityNode for improved scene compatibilities
Improve node serialization/deserialization to be able to recreate the exact same node in the graph when loading a meshroom project, even if the corresponding node's description has changed or does not exist anymore. This allows to recover already computed data on disk, without being impacted by changed uids. CompatibilityNode also provides an on-demand upgrade system to turn into a Node that meets the current node description (if possible).
 
* new abstract class BaseNode, base class for Node and CompatibiliyNode 
* Node: serialize everything needed to spawn a CompatibilityNode with precomputed outputs: inputs, uids, parallelization settings, unresolved internal folders and outputs
* node_factory: handles node deserialization and compatibility issues to create either a Node or a CompatibilityNode
* add compatibility unit tests
2018-07-19 13:50:45 +02:00
Yann Lanthony
b6cbb0cc63 [desc] fix default values for List/GroupAttribute 2018-07-19 13:50:45 +02:00
Yann Lanthony
67ac4bf2ca [core] split uids computation + cache override in updateInternals
* add _computeUids internal method + store computed uids in a dedicated dict
* cache override:
  * add optional cacheDir parameter on updateInternal method to override graph cache directory
  * _buildCmdVars always update member variable '_cmdVars' 
  * CameraInit: ensure node does not belong to a graph and use updateInternals with temp cache directory
2018-07-19 13:50:45 +02:00
Yann Lanthony
727950afef [core] take attributes name into account when computing node's uids
two different attributes with the same value must participate differently to the computed uid
2018-07-19 13:50:45 +02:00
Yann Lanthony
a9e61108ab [core][io] save header with versions metadata in graph file
* save a header with versions metadata:
   * meshroom release 
   * file format
   * node types used in the graph
* save the graph in a dedicated "graph" category
* multiview: store pipeline version in graph header
2018-07-19 13:50:45 +02:00
Yann Lanthony
5f3ddffbbf [core] add 'unregisterNodeType' function 2018-07-19 13:50:45 +02:00
Yann Lanthony
95ca93c977 [core] add utility functions to get nodes and modules versions 2018-07-19 13:50:45 +02:00
Yann Lanthony
3129e5bff9 [core] variable renaming in loadPlugins function
remove references to "node"
2018-07-19 13:50:45 +02:00
Yann Lanthony
5d14189206 [graph] convert cache directory to unix-style filepath in cacheDir setter 2018-07-19 13:50:45 +02:00
Yann Lanthony
346277629a [nodes] use 'desc.Node.internalFolder' in outputs values
* avoid duplicating the same string everywhere
* don't override 'internalFolder' with the same value default value in subclasses
2018-07-19 13:50:45 +02:00
Yann Lanthony
1f675a0e42 [core] split graph.py into separate modules
core.graph is now splitted into: 
  * graph.py
  * node.py
  * attribute.py
2018-06-26 12:19:14 +02:00
Yann Lanthony
3ad2146e06 [core] remove links when clearing ListAttribute 2018-06-18 14:58:37 +02:00
Yann Lanthony
4d8a177ae3 [core] ListAttribute: don't call 'validateValue' on links
Links (Attribute reference or link expression) should not go through value confirmity checker
2018-06-18 14:58:36 +02:00
Yann Lanthony
168866aed0 [core] rename 'graph.isLink' to 'graph.isLinkExpression'
avoid confusion with Attribute.isLink method
2018-06-18 14:58:36 +02:00
Yann Lanthony
99d6068250 [graph] make ListAttribute connectable at root level
ListAttribute can now be connected :
 * at root level to another ListAttribute (new)
 * at child level to basic Attributes
2018-06-18 14:58:35 +02:00
Yann Lanthony
2cd724f957 [graph] ListAttribute: replace '__getitem__ by 'at'
__getitem__ is spuriously called when the object is used inside a Qt model and exposed to QML, leading to unexpected behaviors
2018-06-18 14:43:18 +02:00
Yann Lanthony
94b895ae7c [core] use logging to report plugin loading errors 2018-06-06 11:47:52 +02:00
Yann Lanthony
773d9a6eed [core] use pkgutil to discover packages and modules
* avoid manually finding .py files
* abstract finding of .py/.pyc module files
2018-06-06 11:46:45 +02:00
Yann Lanthony
e2c5a916b5 [core] kill process tree when aborting computation
fix defunct child processes
2018-05-28 16:25:00 +02:00
Fabien Castan
d83022714f [core] add pyCompatibility module
For python 2 / 3 compatibility
2018-05-21 17:12:22 +02:00
Fabien Castan
fb549ff3df [python] use items instead of iteritems for python2/3 compatibility 2018-05-21 16:13:43 +02:00
Yann Lanthony
8466264cf0
Merge pull request #106 from alicevision/dev_retrocompatibility
Retrocompatibility V1
2018-04-16 18:39:32 +02:00
Yann Lanthony
768905d9e9 [nodes] aliceVision: remove '__version__' used by rez 2018-04-13 22:43:00 +02:00
Yann Lanthony
0adc4d8cc6 [core] add low-level retro-compatibility for attribute changes
First version of retrocompatibility, allowing to load files referencing removed or type-incompatible attributes.
* add node_factory to centralize node instantiation
* discard invalid attributes (i.e. not part of the node description anymore or with incompatible value type) when loading a file
* raise on unknown nodes
* add 'core.exception' module to declare Meshroom's exception types
2018-04-13 22:10:13 +02:00
Yann Lanthony
f401ca7c8b [core] fix and use 'validateValue' for ListAttribute and GroupAttribute 2018-04-13 21:51:13 +02:00
Yann Lanthony
7da3f36442 [core] setup low-level logging 2018-04-13 20:42:44 +02:00
Yann Lanthony
12d62fe46b [core] fix ChoiceParam.validateValue
return value(s) converted to the expected type
2018-04-04 19:24:41 +02:00
Yann Lanthony
a677f6b297 [core][win] remove non available stat "num_fds" 2018-03-21 17:24:19 +01:00
Yann Lanthony
8ed0d0a7a2 [graph] add 'asLinkExpr' method 2018-03-21 15:53:34 +01:00
Yann Lanthony
995c18961b [graph] init chunks upon node creation 2018-03-21 15:53:34 +01:00
Yann Lanthony
9522ea7092 [core] use string formatting to print command info 2018-02-26 12:23:08 +01:00
Yann Lanthony
39abaf657a [graph] add 'isLink' helper method
evaluates if the given value matches the link expression pattern
2018-02-12 17:58:47 +01:00