Fabien Castan
1cd4034784
[core] status: add packageName, packageVersion, hostname, sessionUid
2017-11-23 20:07:06 +01:00
Fabien Castan
c677243f90
[core] fix status check: should be done on chunk level (and not on node level)
2017-11-23 16:45:02 +01:00
Fabien Castan
50a4330b10
[core] fix upgrade status of chunks
2017-11-23 16:43:12 +01:00
Fabien Castan
c57d7ed37e
[core] add start/end datetime and elapsed time in status file
2017-11-22 15:17:39 +01:00
Fabien Castan
07351c911a
[core] do not change status on chunks with SUCCESS status
...
If the node is only partially computed, do not change the status of
chunks with SUCCESS status.
2017-11-22 15:16:57 +01:00
Fabien Castan
e3815f74c7
[core] check RUNNING status in chunk compute function and add option to force computation
2017-11-21 12:24:16 +01:00
Yann Lanthony
851ebf60ee
[core] add support for external graph computation via 'Submitter' API
...
* defines a base class for Submitters
* add method in graph module to submit a graph computation to a Submitter
* add SimpleFarmSubmitter and generalize meshroom_submit binary
2017-11-17 19:17:36 +01:00
Yann Lanthony
e8c579c942
[core] parallelization: introduce the notion of node 'size'
...
* node size is an estimation of the number of element to process on a given node, that may be used for parallelization.
* replace the explicit reference to an attribute on a node (Parallelization.inputListParamName)
* size can be:
* dynamic: depends on the size of the node connected to a specific Attribute
* static
2017-11-17 18:46:21 +01:00
Yann Lanthony
877025090e
[graph] fix flowEdges method
...
only consider edges returned by dfsMaxEdgeLength
2017-11-17 18:38:44 +01:00
Yann Lanthony
900516a854
[graph] GroupAttribute: iterate over items() for uid generation
...
__iter__ of core/qt backends differs for DictModel, which lead to inchorent uids.
TODO: fix that by defining a real DictModel in qt backend
2017-11-17 18:11:57 +01:00
Yann Lanthony
7d4c006e68
[graph] List: multiple children attribute deletion + remove edges if any
2017-11-14 16:35:24 +01:00
Yann Lanthony
0fa23edc82
[graph] trigger update when ListAttribute changes
2017-11-14 16:34:32 +01:00
Yann Lanthony
5fc1d92cc2
[graph] remove call to node description's updateInternals
...
causes possible undefined behaviors if the graph is modified during this update
2017-11-10 18:21:59 +01:00
Yann Lanthony
bf747f69d6
Merge remote-tracking branch 'origin/develop' into dev_memoryManagement
2017-11-10 11:43:20 +01:00
Fabien Castan
224cd935f9
[graph] only save input params
2017-11-10 11:32:59 +01:00
Yann Lanthony
80cad9386a
[core] handle unsatisfied node parallelization requirements
...
* catch error in case the required attribute is not found in preceding nodes
* set an empty list of chunks when this happens
* avoid uncatched exceptions in the middle of an update
2017-11-08 18:41:34 +01:00
Yann Lanthony
e9e15d7a8b
[graph] fix StatusData.statusName method
2017-11-08 18:13:55 +01:00
Yann Lanthony
9860828c67
[core] fix Graph.stopExecution
...
iterate over and terminate running chunks
2017-11-08 18:12:03 +01:00
Yann Lanthony
e472d0e1fa
Merge remote-tracking branch 'origin/develop' into dev_memoryManagement
2017-11-07 20:56:51 +01:00
Yann Lanthony
87a81ec3fb
[graph] call applyExpr when adding elements to ListAttribute
...
* automatically (re)create edges if any when dynamically adding elements to the list
* don't do anything if the node does not belong to a graph yet
2017-11-07 20:27:03 +01:00
Yann Lanthony
eccef6bed8
[graph] delete all Node edges in removeNode
...
Delete all edges arriving to and starting from this node. This includes edges connected to sub-attributes (children of List/GroupAttributes).
2017-11-07 20:22:34 +01:00
Yann Lanthony
e38f112c55
[core] fix memory management
...
* parent Nodes, Edges and Attributes to their respective models
* use weakrefs for those objects to avoid cyclic references
* add 'root' property on Attribute for parenting to List/GroupAttribute (parent still exists for Qt-style parenting)
* UI: update commands to match those changes
2017-11-07 20:18:17 +01:00
Fabien Castan
1e4f8f8a61
Add chunk notion for parallelization and implement specific updateInternals in CameraInit node
...
* Add chunk notion for parallelization
* Allows Node desc to implement custom updateInternals
* CameraInit node implement a specific updateInternals to update the
input image list
* FeatureExtraction, FeatureMatching, DepthMap, DepthMapFilter:
implement parallelization
2017-11-07 15:47:14 +01:00
Fabien Castan
1472baa477
[core] Graph.update fix: return when the update is disabled
2017-11-06 20:49:07 +01:00
Fabien Castan
0847a80bdc
[core] Attribute: add isDefault and getPrimitiveValue
2017-11-06 20:46:59 +01:00
Fabien Castan
3a6473b78f
[core] Attribute: getExportValue give the expression on output attributes
...
On output attributes, we export the desc value which contains the
expression and not the locally evaluated value.
2017-11-06 20:46:26 +01:00
Fabien Castan
ee7c0d0fd5
[core] Attribute: change isLink check for _applyExpr()
2017-11-06 20:41:29 +01:00
Yann Lanthony
95471e3e1b
[graph] parent root Attributes to the Node
...
Guarantee Python ownership on objects when using Qt backend. Otherwise, the ownership could be transferred to QML when getting objects via a Slot, leading to their destruction at gargabe collection.
2017-11-02 11:12:14 +01:00
Fabien Castan
9d3f7252c1
[core] move process code into the node description
...
Now CommandLineNode contains the code to launch the command line as a
subprocess.
2017-10-31 21:28:44 +01:00
Fabien Castan
eec52e45e3
[core] implement _applyExpr for List/GroupAttributes
2017-10-31 20:55:56 +01:00
Yann Lanthony
fbebecd463
[graph] default uidIndex value for Attribute.uid method
...
This index only makes sense on ListAttributes and GroupAttributes
2017-10-31 19:50:29 +01:00
Yann Lanthony
16e8037fd7
[core] change Attributes declaration on Node description
...
Inputs and outputs Attributes are now declared in two separate lists
* no more isOutput field on Attribute description
* guarantee attributes order (was not the case in Python2 with class attributes)
* Attribute's name is now part of its description
Note: for now, input and output Attributes are still stored in a single model inside a Graph.
2017-10-31 19:47:32 +01:00
Fabien Castan
329dcba3a5
[core] add validator per attribute type
2017-10-31 16:50:31 +01:00
Fabien Castan
46f6865a25
[core] implement uid computation for ListAttributes and GroupAttributes
2017-10-31 10:46:32 +01:00
Yann Lanthony
f1fdd34b6c
[graph] Attribute: robust UIDs for list of elements as value
...
Fix variable UIDs for non-exclusive choices, which could be affected by different string encodings
2017-10-30 19:23:50 +01:00
Yann Lanthony
7e2d37c819
[graph] fix Node status update from cache
...
updateStatusFromCache reset Node's status to NONE is status file does not exist, but does not save it on disk anymore.
2017-10-30 19:18:41 +01:00
Yann Lanthony
f051148d7a
[graph] update cacheDir property on load
...
use property setter instead of changing the value internally
2017-10-30 17:08:25 +01:00
Yann Lanthony
5e4b26fa5e
[graph] automatically trigger Graph updates
...
* update graph internals / node status when:
- an input attribute is set
- edges are added/removed
* add GraphModification context manager to group graph updates
2017-10-30 16:37:09 +01:00
Yann Lanthony
a0cdc65143
[core] add isInput property in Attribute desc
2017-10-30 16:21:51 +01:00
Yann Lanthony
b22c2b5a62
[graph] minor clean
2017-10-30 11:56:29 +01:00
Yann Lanthony
454b19ed0f
[graph] dummy override of List/GroupAttribute uid methods
...
TODO: implement the real uid mechanism based on child attributes
2017-10-30 11:40:40 +01:00
Yann Lanthony
263ad6b4f5
[graph] remove uid on node, use name as unique identifier
2017-10-30 11:38:42 +01:00
Yann Lanthony
c1a0f43ee3
[graph] handle invalidation value for output attributes
...
output attributes UIDs are only based on the hash of their values without the cache folder
2017-10-30 11:35:07 +01:00
Fabien Castan
6f2ea4c8ef
[core] support killed job
...
* add KILLED status
* register atexit function to update the status of running nodes to
KILLED
2017-10-27 11:24:12 +02:00
Fabien Castan
99ec7ab386
[doc] cosmetic
2017-10-26 18:47:27 +02:00
Fabien Castan
486d1081c8
[core] if rez is in env use it for command line creation
2017-10-26 18:47:01 +02:00
Fabien Castan
0b0ef01997
[core] add package and package version notion
2017-10-26 18:02:28 +02:00
Fabien Castan
40974149ee
[core] cache directory per graph
2017-10-26 12:22:06 +02:00
Fabien Castan
e533c588dd
[core] graph: add the notion of flowEdges and minMaxDepthPerNode
...
- flowEdges: Compute a transitive reduction of the graph to provide
node-level minimal dependencies to create an execution workflow for
renderfarm.
- minMaxDepthPerNode: compute the min and max depth per node
2017-10-24 20:22:39 +02:00
Fabien Castan
5d8771f07e
[core] graph: retrieve nodes and edges at the same time
2017-10-24 20:19:24 +02:00
