RFC Complex access rights for records

Current (legacy) situation
On Invenio 1.X records are restricted based on collection rights (if the record is not in any collection it is restricted to the owner and the admins).
Also there is the possibility to grant access to a specific record inside a restricted collection by using certain MARC tags, i.e 506
Migration problem
Sometimes two collections were created artificially to restrict a set of records, i.e. "Videos" and "Restricted Videos".
Moreover in the situation where one gives access to a record inside the restricted collection to a given person, this person can't search for the record she/he is allow to see inside the restricted collection. This kind os technic allows the users to see the record only if they know the URL.
This artificial separation between public and restricted records can now be achieve using facets.
Proposal
Store the access rights inside the metadata of the record, i.e.
{
  "_access": {
    "read": {
      "system": ["group_a", "group_b"],
      "user"<EMAIL_ADDRESS>    }
  }
}

The so called system field stores the access rights derived by the system from, for example, the collections/communities that the record belongs to. This field should not be modified by anyone except by the system.
On the other hand the user field is meant to be modify by the users, for example upon submission, so the user submitting something can decide to share her/his work with some else which is not part of the community.
If both fields are empty, the record is public.
Possible problems with this solution

Potentially the records will get modify just to change the access rights and this will generate unnecessary versions.
If we combine this solution with collections (or any other thing defining access rights) we need to be sure the access rights are correctly propagated to the record.

Possible benefits of this solution

Potentially the records will get modify just to change the access rights and this will generate a useful trace of the evolution of the access rights (maybe easier to combine with memento API if we want to expose all the versions).
Makes migration much easier eliminating the headache of cleaning the collection tree and the access right definitions for viewrestrcoll action.
Access rights are defined in one place, so answering questions like "why don't I have access to this record?" are much easier to answer.

We have made a small proof of concept on the CDS overlay
Counterproposal
Migrate the viewrestrcoll action as it is, creating a parametrised action, and when migrating the records also migrate the collections they belong to so they can be used to decide whether a user has access to a given record or not.
Possible problems with this solution

The "user" access rights definitions have to be migrated and treated in the same way as in the first proposal.
All the access rights definitions are going to be migrated, meaning a cleanup in the current system needs to be done (always talking from the point of view of v1.2 -> v3.0 migration)

Possible benefits of this solution

Access rights don't need to be propagated to the record.

Note for the reader
The RFC might seems "migration driven" but, regardless if one starts clean or from an existing system, the way access rights are defined IMHO should be the same for the sake of everyone 😉
CC @inveniosoftware/triagers @omelkonian

Storing ACL in records has many benefits indeed.  Two questions:

What about serialisation/deserialisation?

Would you keep ACL completely separated from metadata in a separate input/output section, using our own ACL language?
Or would you try to map them into "master format" ACL language if possible, say MARC21 field 540 and friends?

(E.g. thinking of a use case when a cataloguer may want to edit ACL rights when editing a record in the separated MARC editor such as used by TIND. CC @audub @Kennethhole @PXke)

When musing about these topics, it may be useful to ponder W3C WebAccessControl practices that e.g. Fedora Commons uses.

In our current system we do access rights by a lot of collection juggling. That is, even today we have access rights in the record (sort of), however in a quite obscured way as one needs to fiddle out what rights result from the intersection of the collections exposed in the bibliographic. Indeed, even today this results in quite a bunch of versions that result only from fixing access rights. (Point is, that in our setups at join² it is a requirement that records are not publicly available till they reach a certain curation stage.)
Right now, all this is done by using websubmit with quite a bit of program logic that is not accessible to the cataloguers + the restriction to most of the cataloguers to use websubmit only to ensure all rights are handled properly.
Given this, I follow @egabancho argument. I would argue that access rights should live in the record from the start in a clear manner, replacing the current collection handling by an easier readable method. Furthermore I do not see the drawbacks outlined by @egabancho as such grave issues while I quite agree with the benefits he pointed out above. Even a history of changes that stem only from access right handling is probably not that much of an drawback. Probably the largest issue is the outlined forced compliance with collections, however this should happen anyway.
Concerning serialization: if one want to give cataloguers an easy view and access to the ACLs I think mapping it to the master format is imperative. From our experience I'd strongly argue to give cataloguers access to the ACL. (Maybe this can be restricted to a sub set of specially privileged cataloguers, if in doubt, but it should not require programmers or admins.) In fact we wrote quite a bit of websubmit-code to make this sort of possible. We'd happily drop this code ;)
Some thoughts:

What happens on exchange of records? Ie. how to ensure that rights are propagated properly. In 1.x InstallInvenio ensures in join² that all partners have the same collections with the same access rights to them. As access is driven by collections we just leave them untouched thus the records get proper rights. If I understand @egabancho proposal correctly this would not be necessary anymore and everything would just work. I'd consider this an advantage.
One should probably consider using sort of templates for ACL assignments. To hook it up with cataloguing tools for cataloguers a simple association of an authority record comes to mind. E.g. I could well imagine enhanding 540 by the currently still missing $0 to assign an authority ID. Effectively, this linked record can contain all the technical stuff like static firerole definitions.
One should consider to handle files similarly to the bibliographic. E.g. if one goes for som 5xx in Marc to handle bibliographic ACL one should consider going for holdings in 8xx instead of bibdocfile to handle files and ACLs.

For me the proposal of @egabancho looks like the way to go. And I don't think the "problem" of modified records on change in access rights is a problem at all. I would say it is good preservation and one can easily see who has had access to the record in the past as well.
We have not decided on how we will deal with ACL in the MARC editor yet, but I am pretty sure we will end up with another way than setting it in the MARC tags in the editor.

I am pretty sure we will end up with another way than setting it in the MARC tags in the editor.

I admit I'd prefer to see it "like any other field". In the past, especially when it came to invenios file handling, not having it in the Marc was always one of the major drawbacks we experienced.

If we go with storing ACL in the record, then the implementation should be agnostic to your preferred data model (i.e. something along the lines in the example). However, it should be fairly straight forward to provide some extra dojson rules in Invenio-MARC21 which exposes the ACL in MARC.

When musing about these topics, it may be useful to ponder W3C WebAccessControl practices that e.g. Fedora Commons uses.

👍

If we go with storing ACL in the record, then the implementation should be agnostic to your preferred data model

Agree. Just %s/Marc21/your data model/g and %s/bibedit/your data editor/g.
Just bear with me poor librarian who only has The Data Model(tm) (singular) that for some reason it is called Marc21. Nothing else relevant or in sight for the forseeable future.
What I wanted to say is that the fancy interfaces that are not accessible in your data editor are basically inaccessible in our experience.
Additionally, my librarians tell me that editing Marc is fast but clicking through buttons and masks usually is slow. So I'd strongly vote for making stuff available through the editor in their usual language. (Most of the time this is just a MLE element showing Marc tags and subfields in free form and some non-proporional font.)

First a question: If this design is accepted, would it replace the existing code or be an alternative implementation, giving to overlay developers the possibility to choose one or the other?
And 3 different notes:
1/ It seems to me that this proposal describes how the permissions are represented in the record, but not how they would be checked in Invenio-access's code.
We have a requirement for B2Share which will probably be overkill for most other Invenio services but maybe it can work with the design suggested by @egabancho.
In B2Share some actions on the REST API can require one or more permissions depending.
Example:
We have a publication status field in our deposits. To publish a record the user changes this field's value from draft or submitted to published. This will trigger the creation of the final record.
Thus a PATCH action can either:

modify the deposit's metadata
change the state of the deposit.
do both
And the tricky part is that the permissions are not the same for changing the state of the deposit and to modify the its metadata. A user can have one and not the other, none or both.

2/ @aw-bib I see one problem if the users start modifying permissions: access rights are given for specific reasons. Sometime multiple reasons give the same access right (example: user is owner and is in a specific group of users). By manipulating permissions instead of the original reasons, the result can be extremely difficult to understand or change programmatically (ex: during migrations).
3/ Another thing which could be useful, but is more a nice to have, is to somehow be able to recompute why a given user has a given permission.
Here is an example of how salesforce UI looks like when explaining permissions.

When musing about these topics, it may be useful to ponder W3C WebAccessControl practices that e.g. Fedora Commons uses.

👍

1/ It seems to me that this proposal describes how the permissions are represented in the record, but not how they would be checked in Invenio-access's code.

@nharraud we have a small proposal in our overlay, is this what you meant?

@nharraud

I see one problem if the users start modifying permissions: access rights are given for specific reasons.

Sure.  I do not think about exposing access rights to every user as I would not expose bibedit to them. However, I have three kinds of users in mind (actually we have four).

Real Users. In our case at join² these are scientists, secretaries and so on. People who originally submit stuff to the database by some fancy web form (aka websubmit or webdeposit etc.)
Priviledge users. These people still only have the same ability as users, but they can approve a record to make it public.
Librarians. So to say second level users. Trained in what they do in cataloguing.
Programmers.

The number of people in these ranks decrease dramatically from top to bottom.

By manipulating permissions instead of the original reasons, the result can be extremely difficult to understand or change programmatically

I see your point. Basically what I have in mind is in our workflow:

websubmit by users: they see a predefined form that hides all the technicalities. They only get a pretty limited subset of valid fields and there is a programmatic logic that would assign rights to the records and stuff. (At join² this is basically done by websubmit-functions that implement what upstream would become the workflow.) These users would never ever get an interface to change permissions or do complex things beyond what is allowed in the GUI. They live in the GUI: they may use any button or field, but they are restricted to the buttons and fields. So: all rights assignments and what not is done programmatically.
The same is true for the more priviledge users. Just the automatically assigned rights change.
My librarians act as sort of privileged users as well. They use this very same GUI for >90% of their tasks, ie. they see nothing else than the normal user. They do the same things, it's just in our worflow their "submit" again changes some access rights (e.g. they can approve an attached document to be public.)

However, for this <10% of their tasks, ie. once my fellow libraians (and only they have access to it) open bibedit I would like to give them the possibility to see and assign rights if needs be. I see your point that this may get messy, however, at this stage I have trained people who know what they do. And they'd usually only fix some issue that arose.
In my world bibedit is the low level editor and it should allow low level by having access to the necessary stuff. So, Basically, I'd like to see sort of full access on the record level in an interface easier than ipyhton.
Why? There are tasks I could easily give to a librarian to fix that otherwise require a programmer to the same end. We are always quite a bit short of the latter. (Not that we have too many librarians either.) So, it can be much easier and faster to fix 500 records manually than getting someone writing the python for it. (Frankly: till about ~1000 records I tend not to write code, they are faster.) If there is an interface they understand. One of the worst issues in Invenio1.x bibedit e.g. is it's entire inability to handle bibdocs and their permissions. By assigning records to collections we can work around the limitations of not accessing the rights directly. However, it is as @egabancho describes, sometimes quite obscure what rights result from which assignment.

@egabancho What is the status of this issue?
Would this new feature be added to invenio-access or in a separate module?

I'm adding milestone v1.1 to deal with this shortly after a first final v3 release. In principle this is not such much about Invenio-Access but more about defining actions and default permission factories in Invenio-Records.


Add XDebug config to support php debugging.

Changes

Added XDebug configuration to support remote debugging of php code

We recently upgraded to PHP8.4 and changed our Dockerfile a lot, can you please ajust this PR with new changes. If you do not have time to do this, please let me know so I can close this PR.

Sorry for taking too long. Using XDebug is something new for me and I didn't know what capabilities there are and I currently don't have too much time figuring it all out.
Thanks for your patience. The PR can be closed. I will open up a new one if I really have more time for that. Or maybe someone else will have it done by then.


Make referenced XMLHelper consistent

See https://github.com/TU-Berlin/mathosphere/issues/53

Changing this to move XMLHelper into MathMLQueryGenerator


fix: mobile video sizing

Description
fixes: https://www.notion.so/oaknationalacademy/Mobile-video-sizing-aad5baea2dd14bfb856506ed194419d6

:tada: This PR is included in version 1.7.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Converts Services to Ember.Service extensions

Injects services via Ember.inject instead of initializer
Removes Facebook facepile, Facebook deprecated/removed it
Fixes tests

Tests intermittently failed locally

Just confirm, Facebook's Facepile stopped working June 23rd, 2015.
The failing tests on Travis should be because it is still building with Phantom 1.9. Rebasing this PR should probably fix it. I'll do a rebase this side as well to see.

@kennethkalmer What branch should this be rebased against? I'm not seeing anything with Phantom 2.0

I'm seeing deprecations in Ember 1.13.x and this pull request has the fixes, is there anything I can do to get it over the line? The PR looks correct.


Hound seems to ignore max_line_length setting in coffeescript_style.json

We just activated Hound and it raised a few "Line exceeds maximum allowed length" errors where the lines are shorter than the maximum we set.
In the coffeescript_style.json we have the following:
  "max_line_length": {
    "level": "error",
    "value": 120
  }

And here an example of Hound complaining about a line which is only 95 characters long:

I have seen that there was an issue like this before (https://github.com/thoughtbot/hound/issues/132), but it was resolved.

Using coffeelint gem and running it from the console works fine though. No errors about max line length.

I assume you have instructed Hound to read coffeescript_style.json in .hound.yml? Remember that both coffeescript_style.json and an updated .hound.yml need to be present in the PR branch for the new setting to take effect.

I just found the cause: the file names inside .hound.yml have dashes as separators while the actual files have underscores.
Sorry...

@codejet, no problem. Thanks for letting us know and closing the issue.


Add collect_runs() function

This commit adds a new algorithms function, collect_runs(), which is
used to find a list of runs in a PyDiGraph object that match a specified
condition. A run in this context is any linear path inside the graph.
The condition is specified using a python function that will be passed a
node's data payload/weight object and will return whether it matches the
condition or not. The intended use case for this function is with
qiskit-terra's PyDiGraph method collect_runs() which is implementing
this algorithm currently in Python on top of PyDiGraph (it will also be
used by a future method collect_1q_runs which is being added in
Qiskit/Qiskit-terra#5431).
TODO

[ ] Add tests

Pull Request Test Coverage Report for Build 383885846

2 of 38 (5.26%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.08%) to 93.635%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/lib.rs
2
38
5.26%

Totals

Change from base Build 379386770:
-1.08%

Covered Lines:
2957

Relevant Lines:
3158

💛  - Coveralls

I think there are also some edge cases here when I ran qiskit-terra's tests with:
index d56ca28d..a1480a5b 100644
--- a/qiskit/dagcircuit/dagcircuit.py
+++ b/qiskit/dagcircuit/dagcircuit.py
@@ -1378,29 +1378,13 @@ class DAGCircuit:

         Nodes must have only one successor to continue the run.
         """
-        group_list = []
-
-        # Iterate through the nodes of self in topological order
-        # and form tuples containing sequences of gates
-        # on the same qubit(s).
-        topo_ops = list(self.topological_op_nodes())
-        nodes_seen = dict(zip(topo_ops, [False] * len(topo_ops)))
-        for node in topo_ops:
-            if node.name in namelist and node.condition is None \
-                    and not nodes_seen[node]:
-                group = [node]
-                nodes_seen[node] = True
-                s = self._multi_graph.successors(node._node_id)
-                while len(s) == 1 and \
-                        s[0].type == "op" and \
-                        s[0].name in namelist and \
-                        s[0].condition is None:
-                    group.append(s[0])
-                    nodes_seen[s[0]] = True
-                    s = self._multi_graph.successors(s[0]._node_id)
-                if len(group) >= 1:
-                    group_list.append(tuple(group))
-        return set(group_list)
+
+        def filter_fn(node):
+            return node.type == "op" and node.name in namelist and node.condition is None
+
+        group_list = rx.collect_runs(self._multi_graph, filter_fn)
+        print(group_list)
+        return set(tuple(x) for x in group_list)

     def nodes_on_wire(self, wire, only_ops=False):
         """

applied most of the tests work, but there are still some failures caused by differences in the returned runs between this function and what was in terra before.

Ok, I fixed the bug the terra unit tests identified in: https://github.com/Qiskit/retworkx/pull/206/commits/4c5d8ebe806f344a34748893e8ee986d1fc839cf and also added some initial tests in https://github.com/Qiskit/retworkx/pull/206/commits/be9c548bb02b3d9fee324b7ddd0c6eb1adcdc803 I'll probably add a couple more negative tests depending on what coveralls shows, but this should be ready to go now.

I understand what is strange for the original implementation in qiskit-terra.
The original one can report a node in multiple runs.
For example, it reports cx twice, {('h', 'cx'), ('h', 'cx')}, for the circuit below.
┌───┐
q_0: ┤ H ├──■──
├───┤┌─┴─┐
q_1: ┤ H ├┤ X ├
└───┘└───┘
The collect_run() implemented in this PR prevents such a duplication, and reports
[['h', 'cx'], ['h']] for the above case.
I confirmed that this change does not affect to current DAGCircuit usages within terra, collect_run is only used to collect consecutive CNOTs or 1q-gates. So I have little concern on the spec change induced by this PR.
I like the spec implemented in this PR (it's clearer and simpler), so I suggest to

clearly state the point above (no duplication) in the docstring, e.g. Each node can belong to at most one run.
add the above case to the test (and two more as below, if you don't mind)

    def test_h_h_cx(self):
        dag = retworkx.PyDiGraph()
        q0, q1 = dag.add_nodes_from(['q0', 'q1'])
        h_1 = dag.add_child(q0, 'h', 'q0')
        h_2 = dag.add_child(q1, 'h', 'q1')
        cx_2 = dag.add_child(h_1, 'cx', 'q0')
        dag.add_edge(h_2, cx_2, 'q1')

        def filter_function(node):
            return node in ['cx', 'h']

        res = retworkx.collect_runs(dag, filter_function)
        self.assertEqual([['h', 'cx'], ['h']], res)

    def test_cx_h_h_cx(self):
        dag = retworkx.PyDiGraph()
        q0, q1 = dag.add_nodes_from(['q0', 'q1'])
        cx_1 = dag.add_child(q0, 'cx', 'q0')
        dag.add_edge(q1, cx_1, 'q1')
        h_1 = dag.add_child(cx_1, 'h', 'q0')
        h_2 = dag.add_child(cx_1, 'h', 'q1')
        cx_2 = dag.add_child(h_1, 'cx', 'q0')
        dag.add_edge(h_2, cx_2, 'q1')

        def filter_function(node):
            return node in ['cx', 'h']

        res = retworkx.collect_runs(dag, filter_function)
        self.assertEqual([['cx'], ['h', 'cx'], ['h']], res)

    def test_cx_h_cx(self):
        dag = retworkx.PyDiGraph()
        q0, q1 = dag.add_nodes_from(['q0', 'q1'])
        cx_1 = dag.add_child(q0, 'cx', 'q0')
        dag.add_edge(q1, cx_1, 'q1')
        h_1 = dag.add_child(cx_1, 'h', 'q0')
        cx_2 = dag.add_child(h_1, 'cx', 'q0')
        dag.add_edge(cx_1, cx_2, 'q1')

        def filter_function(node):
            return node in ['cx', 'h']

        res = retworkx.collect_runs(dag, filter_function)
        self.assertEqual([['cx'], ['h', 'cx']], res)

The above correspond with the circuits:
     ┌───┐
q_0: ┤ H ├──■──
     ├───┤┌─┴─┐
q_1: ┤ H ├┤ X ├
     └───┘└───┘

          ┌───┐
q_0: ──■──┤ H ├──■──
     ┌─┴─┐├───┤┌─┴─┐
q_1: ┤ X ├┤ H ├┤ X ├
     └───┘└───┘└───┘

          ┌───┐
q_0: ──■──┤ H ├──■──
     ┌─┴─┐└───┘┌─┴─┐
q_1: ┤ X ├─────┤ X ├
     └───┘     └───┘

I'm not sure it helps users to add circuit diagrams in docstring since they don't have information on edge directions.

I understand what is strange for the original implementation in qiskit-terra.
The original one can report a node in multiple runs.

Oh, that's interesting, I'm actually a bit surprised I basically just ported the logic from the terra DAGCircuit.collect_runs() method to rust and adapted it to work inside retworkx. I didn't notice this, but also didn't think to test it since the use cases I had in mind were all about the getting the 1q gates runs for the optimize_1q_gates passes.

For example, it reports cx twice, {('h', 'cx'), ('h', 'cx')}, for the circuit below.
     ┌───┐
q_0: ┤ H ├──■──
     ├───┤┌─┴─┐
q_1: ┤ H ├┤ X ├
     └───┘└───┘

The collect_run() implemented in this PR prevents such a duplication, and reports
[['h', 'cx'], ['h']] for the above case.
I confirmed that this change does not affect to current DAGCircuit usages within terra, collect_run is only used to collect consecutive CNOTs or 1q-gates. So I have little concern on the spec change induced by this PR.
I like the spec implemented in this PR (it's clearer), so I suggest to
* clearly state the point above (no duplication) in the docstring, e.g. Each node can belong to at most one run.

* add the above case to the test (and two more as below, if you don't mind)

    def test_h_h_cx(self):
        dag = retworkx.PyDiGraph()
        q0, q1 = dag.add_nodes_from(['q0', 'q1'])
        h_1 = dag.add_child(q0, 'h', 'q0')
        h_2 = dag.add_child(q1, 'h', 'q1')
        cx_2 = dag.add_child(h_1, 'cx', 'q0')
        dag.add_edge(h_2, cx_2, 'q1')

        def filter_function(node):
            return node in ['cx', 'h']

        res = retworkx.collect_runs(dag, filter_function)
        self.assertEqual([['h', 'cx'], ['h']], res)

    def test_cx_h_h_cx(self):
        dag = retworkx.PyDiGraph()
        q0, q1 = dag.add_nodes_from(['q0', 'q1'])
        cx_1 = dag.add_child(q0, 'cx', 'q0')
        dag.add_edge(q1, cx_1, 'q1')
        h_1 = dag.add_child(cx_1, 'h', 'q0')
        h_2 = dag.add_child(cx_1, 'h', 'q1')
        cx_2 = dag.add_child(h_1, 'cx', 'q0')
        dag.add_edge(h_2, cx_2, 'q1')

        def filter_function(node):
            return node in ['cx', 'h']

        res = retworkx.collect_runs(dag, filter_function)
        self.assertEqual([['cx'], ['h', 'cx'], ['h']], res)

    def test_cx_h_cx(self):
        dag = retworkx.PyDiGraph()
        q0, q1 = dag.add_nodes_from(['q0', 'q1'])
        cx_1 = dag.add_child(q0, 'cx', 'q0')
        dag.add_edge(q1, cx_1, 'q1')
        h_1 = dag.add_child(cx_1, 'h', 'q0')
        cx_2 = dag.add_child(h_1, 'cx', 'q0')
        dag.add_edge(cx_1, cx_2, 'q1')

        def filter_function(node):
            return node in ['cx', 'h']

        res = retworkx.collect_runs(dag, filter_function)
        self.assertEqual([['cx'], ['h', 'cx']], res)

Yeah I agree with this, I've done this in: https://github.com/Qiskit/retworkx/pull/206/commits/becd66852b6511e386667e7d9771f1e5e90788f3

The above cases correspond with the circuits, respectively:
     ┌───┐
q_0: ┤ H ├──■──
     ├───┤┌─┴─┐
q_1: ┤ H ├┤ X ├
     └───┘└───┘

          ┌───┐
q_0: ──■──┤ H ├──■──
     ┌─┴─┐├───┤┌─┴─┐
q_1: ┤ X ├┤ H ├┤ X ├
     └───┘└───┘└───┘

          ┌───┐
q_0: ──■──┤ H ├──■──
     ┌─┴─┐└───┘┌─┴─┐
q_1: ┤ X ├─────┤ X ├
     └───┘     └───┘

I'm not sure adding circuit diagrams in docstring helps readers as they don't have information on edge directions.

Yeah I'm not sure if circuit diagrams make sense in the docstring for retworkx tests, I can add them if you think it makes sense. For retworkx functions we can always leverage the graph drawer and jupyter-execute like: https://github.com/Qiskit/retworkx/blob/master/src/generators.rs#L50-L71 That doesn't do anything for tests though.


Enable dlt to run on managed AWS airflow MWAA

Feature description
User:
"I've been at a stand still actually. It looks to me like the constraints from MWAA are going to prohibit me from using Snowflake as a destination. Writing to S3 is a viable replacement- but still having trouble here as well. The most recent version of airflow available on MWAA is 2.7.2.
At the moment, airflow is showing some import conflicts. It can't see the path to locate PipelineTasksGroup in my dag, nor can it find DltResource in init.py.
There is also a version conflict with s3fs which relies on aiobotocore. The constraint on aiobotocore  here is 2.6.0 which is not"
Are you a dlt user?
I'd consider using dlt, but it's lacking a feature I need.
Use case
Run airflow on managed AWS with dlt. Library conflicts, seems S3 could be easier .
You can ask more information from the user here https://dlthub-community.slack.com/archives/C04DQA7JJN6/p1707496343595039

@adrianbr the only way to fix it is to try that ourselves.

there's local runner https://docs.aws.amazon.com/mwaa/latest/userguide/working-dags-dependencies.html https://github.com/aws/aws-mwaa-local-runner/tree/v2.7.2
we need to figure out how to run our helper on it
possibly update our airflow CI workflow to run some dags on MWAA

I might be able to help here. I have been using dlt with MWAA successfully. I'm not writing to Snowflake, but have run into problems with the constraints multiple times.
My workaround was to add --constraints /dev/null to the uploaded requirements.txt file, which overrides the default constraints imposed by Airflow. It seems hacky, but AWS actually encourages this if you don't like the imposed constraints.
Idea came from this article

we had a similar issue on v2.2.2 (--constraints needed in this version though) and decided to install additional packages directly in a virtual environement using the PythonVirtualenvOperator operator. First virtualenv needs to be part of your requirements.txt
https://docs.aws.amazon.com/mwaa/latest/userguide/samples-virtualenv.html


Voyager Error After update from [1.3] to [1.4]

Version information

Laravel: v7.26
Voyager: v1.4
PHP: 7.4.7
Database: [MySQL] [7.4.7]

Description:
GET http://domain.org/admin/voyager-assets?path=js%2Fapp.js net::ERR_ABORTED 500 (Internal Server Error)
admin:277 Uncaught ReferenceError: Vue is not defined at admin:277
(anonymous) @ admin:277 (Voyager Error After update from [1.3] to [1.4])
Additional context
Hello Voyager Artisan  please out with this issue!
after i update  laravel composer and also voyager laravel From version [5.8] to [7.26]
everything works fine except my laravel-voyager not loading properly and spinner keep spinning
I have done everything i can but to no good result
this is the display image below

There are some issue with the exact same problem.
It's very likely a problem with an extension in your browser, another laravel package or the browser itself.
Check if it's working in incognito mode, check your Laravel logs, and try it in a different browser.

There are some issue with the exact same problem.
It's very likely a problem with an extension in your browser, another laravel package or the browser itself.
Check if it's working in incognito mode, check your Laravel logs, and try it in a different browser.

Thank for Repply @emptynick
i have tried it on Opera Browers,Firefox for dev and chrome incognito mode and error still coming out nothing change

And the logs?

And the logs?

(log from firefox)
XHR GEThttps://content.googleapis.com/discovery/v1/apis/analytics/v3/rest?fields=kind%2Cname%2Cversion%2CrootUrl%2CservicePath%2Cresources%2Cparameters%2Cmethods%2CbatchPath%2Cid&pp=0

I mean the Laravel logs.

I mean the Laravel logs.

this is the error from laravel log
[2020-09-12 12:06:11] local.ERROR: Security warning: an embedded

There are two problems here:

The package Efficiently\JqueryLaravel forbids accessing our Javascript file (not sure why it thinks it's responsible to do that)
The package fails because it sees the request for that file coming from another domain. This can be because you are using https and the route to the file is generated as http.

None of those are related to Voyager.


Add support for GPT-4 models

https://platform.openai.com/docs/models/gpt-4

I will update models on next release. You can always pass models as string, you don't need to wait for next release to use GPT-4.

Thank you.


Remove livequery reference

livequery was removed in https://github.com/caioariede/django-location-field/pull/114 but it is still referenced in the docs

Is this project still maintained by anyone?


[bitfinex] Adding withdrawal types missing from current API doc.

Fix for #2308

Some of the added currencies do not exist in org.knowm.xchange.currency, should I add them there also?

yes.


[Idea 💡] Management of multiple solutions

Currently, the ALM strategy proposed in this template repository considered "basically" only one solution.
You can duplicate workflows to manage multiple solutions separately, but I am not sure it is the best way to do it.
With this issue we will try to explore some ways to improve the situation regarding this requirement.

In the GitHub workflow for the export of the solution, we could consider an input of type choice for the solution name:

Source: https://github.blog/changelog/2021-11-10-github-actions-input-types-for-manual-workflows/

Notes:

Identification of considered solution (limit to 1 by process) on issues and pull requests using a tag 🏷 - in the corresponding workflow we will loop on all the tags and search for known solutions (documented in a solutions configuration Json file)
Consideration of a solutions configuration JSON file to store for each solution informations like: type (internal or external), is packing required (external solution when we have the managed packed version available), pack or build (build for PCF solution), depends on (potential solution required for the considered one - limit to one because I think it is better like this but open to discussion on this obviously)
for push to master, find a way to identify the modified solution (list updates in commit and identify solution?)
find a way to import different solutions from a single workflow (call a workflow with inputs using GitHub cli?)
for solutions always required for work (ex: external solutions) the list will be documented in the environment configuration Json file

https://github.com/marketplace/actions/changed-files

After some work on this topic, I think it would be difficult / unwise to try to cover multiple solutions in one development cycle (issue to production) and also external solutions.
Issue scope update:

Choice input for the selection of the solution to export
Dynamic selection of the considered solution for pull request validation, deployment to validation or production based on files changed
Import all solutions "in the right order" (for dependencies management) during workspace initialization - we will need to improve this part in the future because it can be useless to have all solutions from the repository in some scenarios


Clarify how the email alert check works

https://trello.com/c/13JaJ5mQ/203-review-the-email-alert-traffic-dashboard
This clarifies the description of the email alert check to say that it
(currently) works against two mailboxes, not just one.

🚀 🐦 ✉


Update 2c.js

some sanitizing work, the underscore _ is replaced with (_) in the sanitize function to handle specific cases where it might be a reserved keyword or require special treatment

I don't understand how _ =>  -> (_) => changes anything other than syntax? It will never be anything special in JS.

yeahh, but it looks fancy


How to use gradient function in densecrf.pyx？

Hi, I tried to  use the commented out gradient function in densecrf.pxd. Now I can call it in python but it fails and the python stops working. Is there any advice for me to continue or turn to other methods to implement it in a learning?Thank u very much

Hi, I haven't tried this myself, and don't have the time to, so I'm afraid I can't give you guidance. Python stopping to work probably means you have made a mistake in the wrapping somewhere, you need to be careful with types etc. as it's all C code which will simply crash if something is wrong, instead of giving a nice error as we're used to from Python :smile:
If you do succeed, please let me know, or ideally open a PR.


Update CLI extensions available doc

Update CLI extensions available doc.
Triggered by Azure/azure-cli-extensions - TRAVIS_BUILD_ID=394757546
https://github.com/Azure/azure-cli-extensions/commit/ef41893ce100730d0896445b4460ecbcaabb8150

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/azure-cli-extensions-list.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/azure-cli-extensions-list.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[BUG] Large floating point values written as Inf not Infinity in Hive text writer

The following floating point values are written as Inf, not as Infinity by the Hive delimited text writer in spark-rapids:
3.4028236E38
3.4028236e+38
1.7976931348623157E99
1.7976931348623157e+99
1.7976931348623157E308
1.7976931348623157e+308

These values are written as Infinity in Apache Spark 3.2.x. LazySimpleSerDe does not recognize "Inf" as Infinity. So the GPU output is read incorrectly on CPU.
The values are captured in integration_tests/src/test/resources/hive-delim-text/simple-float-values/simple_float_values.hive.txt. The test that reads from this file is left as an xfail test. It would be good to fix the writer for Inf values.

"Inf" seems hardwired into the CUDF machinery for float casts to string.
The expedient change might be to replace these values during write, on the plugin.


Pagination - There is no Previous, First and Last links in fhir bundle json response. Only self and next link is present

https://build.fhir.org/search.html#paginglinks

Hi @aparrot89,
Thank you for reporting this.
Actually, only Next link is implemented.

Do you think it could be implemented?

Hi @aparrot89, We don't currently have this on our roadmap. We will add this to our backlog but we don't have an ETA on when we would resolve.

Hi,
As Mohammed mentioned, we do not support this feature, and it is not prioritized on the roadmap (due to both complexity and performance concerns).
I am closing this issue.


floatingFilterComponentFramework not wiring up correctly due to missing mandatoryMethodList entry

I'm submitting a ...  (check one with "x")
[x ] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[ ] support request => Please do not submit support request here, instead see https://github.com/ceolter/ag-grid/blob/master/CONTRIBUTING.md#question

Current behavior
When setting the floatingFilterComponentFramework property of a ColumnDefinition you get a runtime error: '"this.floatingFilterComp.onParentModelChanged is not a function"
StackTrace:
"TypeError: this.floatingFilterComp.onParentModelChanged is not a function
at FloatingFilterWrapperComp.onParentModelChanged (http://localhost:3000/scripts/vendor.js:29974:33)
at HeaderRowComp.createFloatingFilterWrapper (http://localhost:3000/scripts/vendor.js:20709:31)
at HeaderRowComp.createHeaderElement (http://localhost:3000/scripts/vendor.js:20694:31)
at http://localhost:3000/scripts/vendor.js:20652:47
at Array.forEach (native)
at HeaderRowComp.onVirtualColumnsChanged (http://localhost:3000/scripts/vendor.js:20638:22)
at HeaderRowComp.init (http://localhost:3000/scripts/vendor.js:20606:14)
Expected behavior
The custom floating filter component should render
Minimal reproduction of the problem with instructions
There is a solution to this which I have debugged.
The issue is that when wrapping the React component you use ReactFrameworkComponentWrapper from ag-grid-react.  This uses the "mandatoryMethodList" property from ComponentProvider in ag-grid to decide which methods to wireup from wrapper to the custom React component.
Looking at the setup (in componentProvider.js) currently it is:
        textFloatingFilterComponent: {
            mandatoryMethodList: [],
            optionalMethodList: [],
            defaultComponent: floatingFilter_1.TextFloatingFilterComp
        },

The solution is to add 'onParentModelUpdated' to the mandatory method list for every floating filter component.  I have done this in the Chrome debugger at runtime, and this solves the problem.
What is the motivation / use case for changing the behavior?
Make the floatingFilterComponentFramework property behave as expected.
Please tell us about your environment:

Windows 10, Webstorm, Webpack, Node.js

ag-Grid version: X.X.X
8.2.0

Browser:
Chrome 56.0.2924.87

Language: [all | TypeScript X.X | ES6/7 | ES5]
Typescript 2.1.4

Thanks for reporting - I've raised AG-335 to track this issue our side, but we will update this thread too with updated

Thanks a lot Sean.

any update on this bug? where can i track the update on AG-355

Hi
This has been released as part of 9.0.0 - apologies, you should have received an update here
thanks

@seanlandsman , i still see this issue on 9.0 too.
On the pipeline page, i see this is still in backlog. Can you please help confirm if the fix is in 9.0.

I've taken a look again and see whats happened here - I raised AG-335, but in your earlier comment today you mentioned AG-355, which is what I used to look it up.
AG-335 (the correct reference) is still open - I'm afraid I dont have an update for you on this

ah thanks, my bad. Hoping for a fix for this soon.

I'm facing the same issue. Please update if there is a fix for the same.
Development environment
Windows 7, Visual Studio Code, Webpack, Node.js
ag-Grid version: X.X.X
9.0.3
Browser:
Chrome 57.0.2987.133
Language: [all | TypeScript X.X | ES6/7 | ES5]
Typescript 2.1.4

Hi,
Just to let you know that v10.0.0 has just been released and includes the fix that we discussed in this thread.
Thanks

I'm getting this issue in 10.0.
this.floatingFilterComp.onParentModelChanged is not a function
at FloatingFilterWrapperComp.onParentModelChanged.
As described above, when I add  'onParentModelChanged' to the mandatoryMethodList in componentProvider.js it works.

node_modules\ag-grid\dist\lib\componentProvider.js    9.1.0 or 10.0.0
var ComponentProvider = (function () {
function ComponentProvider() {
}
ComponentProvider.prototype.postContruct = function () {
this.allComponentConfig = {
dateComponent: {
mandatoryMethodList: ['getDate', 'setDate'],
optionalMethodList: [],
defaultComponent: dateFilter_1.DefaultDateComponent
},
headerComponent: {
mandatoryMethodList: [],
optionalMethodList: [],
defaultComponent: headerComp_1.HeaderComp
},
headerGroupComponent: {
mandatoryMethodList: [],
optionalMethodList: [],
defaultComponent: headerGroupComp_1.HeaderGroupComp
},
setFloatingFilterComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilter_1.SetFloatingFilterComp
},
textFloatingFilterComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilter_1.TextFloatingFilterComp
},
numberFloatingFilterComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilter_1.NumberFloatingFilterComp
},
dateFloatingFilterComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilter_1.DateFloatingFilterComp
},
readModelAsStringFloatingFilterComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilter_1.ReadModelAsStringFloatingFilterComp
},
floatingFilterWrapperComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilterWrapper_1.FloatingFilterWrapperComp
},
emptyFloatingFilterWrapperComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: floatingFilterWrapper_1.EmptyFloatingFilterWrapperComp
},
floatingFilterComponent: {
mandatoryMethodList: ['onParentModelChanged'],
optionalMethodList: [],
defaultComponent: null
},
filterComponent: {
mandatoryMethodList: [],
optionalMethodList: [],
defaultComponent: null
}
};
};

@yuanjinyong the above does seem to stop the error message from happening but the parent model does not appear to be getting updated when I fire onFloatingFilterChanged(). Not sure why but I was able to get a functional filter action happening via a combination of your suggestion and just updating the filter via the grid api. Gist: https://gist.github.com/jeffskelton3/7ced212868abb59a783eda7726724e21
Hopefully that will help anyone who, like me, has been stuck for a bit. Thanks!!


ADDING BETTER UI AND ANIMATIONS TO CONTACT US BUTTONS AND MESSAGE BUTTON.

Yes sure @DHRUVKHANDELWAL00  , Open a PR with required screenshots. We wil review and would be happy to merge it!


Support Kerberos authentication for Kudu Connector

When I use presto to connect kudu tables,I can show tables and describe kudu table's structure,But I can not select data from kudu table,there is error as following:
Query 20190802_074604_00006_2g8pc failed: org.apache.kudu.client.NonRecoverableException: Couldn't find a valid master in (<IP_ADDRESS>:7051,<IP_ADDRESS>:7051,<IP_ADDRESS>:7051). Exceptions received: [org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available]
Anyone has occured this problem?

It is not yet supported.

The Kudu connector does not support Kerberos authentication. Someone will need to add it.

Thank you .I see .But now I can't use presto to connect kudu without Kerberos authentication yet.Here is the error:

How can I solve this problem?

The Kudu connector does not support Kerberos authentication. Someone will need to add it.

Could you help me look at it?Here is the kudu.properties:
connector.name=kudu
kudu.client.master-addresses=<IP_ADDRESS>:7051,<IP_ADDRESS>:7051,<IP_ADDRESS>:7051
[root@cdh215 catalog]#

I have resolved the problem, please see at https://github.com/prestosql/presto/pull/3549


RT-1.29: BGP chained import/export policy attachment changes incorpor…

Updated metadata.textproto
vender check included for NOKIA
Added deviations.
Updated chained_policies_test.go
Removed accept route for statement match-policy-v4 & match-statement-v6 as the next statement prefix-set-v4 & prefix-set-v6 will not be applied if accept route is configured.
Refer PR- #2760
Added deviations for prefixset_Mode.
Updated to use batch replace for delete routing-policies.
Updated README.md
policy-result as ACCEPT_ROUTE not required for match-policy-v4 & match-statement-v6 policy.
Refer PR- #2760
"This code is a Contribution to the OpenConfig Feature Profiles project ("Work") made under the Google Software Grant and Corporate Contributor License Agreement ("CLA") and governed by the Apache License 2.0. No other rights or licenses in or to any of Nokia's intellectual property are granted for any other purpose. This code is provided on an "as is" basis without any warranties of any kind."

Pull Request Functional Test Report for #3505 / e57319344c1d9901af0767d004c2931eded92a7d
Virtual Devices

Device
Test
Test Documentation
Job
Raw Log

Arista cEOS

RT-1.29: BGP chained import/export policy attachment

Cisco 8000E

RT-1.29: BGP chained import/export policy attachment

Cisco XRd

RT-1.29: BGP chained import/export policy attachment

Juniper ncPTX

RT-1.29: BGP chained import/export policy attachment

Nokia SR Linux

RT-1.29: BGP chained import/export policy attachment

Openconfig Lemming

RT-1.29: BGP chained import/export policy attachment

Hardware Devices

Device
Test
Test Documentation
Raw Log

Arista 7808

RT-1.29: BGP chained import/export policy attachment

Cisco 8808

RT-1.29: BGP chained import/export policy attachment

Juniper PTX10008

RT-1.29: BGP chained import/export policy attachment

Nokia 7250 IXR-10e

RT-1.29: BGP chained import/export policy attachment

Help

/fptest virtual

Pull Request Test Coverage Report for Build<PHONE_NUMBER>4
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 55.268%

Totals

Change from base Build<PHONE_NUMBER>0:
0.0%

Covered Lines:
1983

Relevant Lines:
3588

💛  - Coveralls

@ampattan , not this does not pass on srlinux container.  see logs from https://github.com/openconfig/featureprofiles/pull/3505#issuecomment-2403364930

/fptest virtual

/fptest virtual

This fails on srlinux.  See logs:  https://gist.github.com/dplore/9ea5bafc71eb026ef8b8925e08f458fc
Please fix and on next submission, include logs showing pass results on your platform

HW log with these changes are avaialbe at - b/380289415.

This fails on srlinux. See logs: https://gist.github.com/dplore/9ea5bafc71eb026ef8b8925e08f458fc

This log is with older image.  Latest image would work fine for these failures. However, considering this is a KNE run, there is an open issue which is getting tracked here - https://github.com/open-traffic-generator/fp-testbed-nokia/issues/30


Small problem with the Get Request header

First of all the server works great with browsers -
I'm trying to make a client with NodeJS and I've spent my whole afternoon trying to figure out what Node doesn't like about the picohttp response.    Node.js has apparently become more picky over time - lots of people are complaining about  it.
But anyway I had to make a couple changes
in HelloWorldRequestProcess, the Reponse string starts out with HTTP 1.1, but it needs to be HTTP/1.1
The 2nd problem which is that the spec says the line endings need to be CRLF.  the line endings I was getting were just LFs.
I'm using VSCode, and since it is a literal string perhaps that makes a difference.
            var responseString ="HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: " + message.Length + "\r\n";
            responseString += "\r\n" + message;

Hey thanks,
This is a server I made just for learning about HTTP requests/responses. It can only handle one request at a time and not suitable for real use.
Hope you are using it only for experimental purpose. :)


neofetch: remove

By soft abandonment.
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Maybe recommend an alternative in the alias message?

How about we don't just straight up remove one of the most well-known and used linux packages
at the very least have a warning for a while before throwing

following the train of logic of ignoring code usage and only worrying about code age and maintenance we should remove almost every single Xorg package

this PR is solely redundant. we don't need to decide what people should be using so aggressively
neofetch works and a lot of people will instinctively put it in their configs, a throw saying "um actually you shouldn't" will just be a thorn in their boot

https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+in%3Atitle+neofetch I don't see this to be a massive maintenance concern either, since I only see one issue listed that was addressed

following the train of logic of ignoring code usage and only worrying about code age and maintenance we should remove almost every single Xorg package

Except Xorg was updated less than a month ago:
https://www.x.org/releases/individual/

Xorg was updated less than a month ago: https://www.x.org/releases/individual/

Many Xorg utilities haven't seen updates in over a decade. Some as long as 15 years ago.

this PR is solely redundant. we don't need to decide what people should be using so aggressively

I thought on the side of Nixpkgs maintenance. We are using patches over the tip of master, after all.

neofetch works and a lot of people will instinctively put it in their configs, a throw saying "um actually you shouldn't" will just be a thorn in their boot

That's a good argument. I will close this, then.

Many Xorg utilities haven't seen updates in over a decade. Some as long as 15 years ago.

this is what i meant

That's a good argument. I will close this, then.
thanks


generate pai_service_name label from app label

make prometheus generate pai_service_name lable for scraped metrics.

Coverage increased (+18.0%) to 65.521% when pulling 2e647b1abaa70407dd61958b13378deb9a03bb21 on dixu/expose-pai-service-name into 17ed648d1bfdab312d555b49278ef98459e39b39 on master.


docs: update readme plugin list

:tada: This PR is included in version 3.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Bump guava from 32.0.0-jre to 32.1.3-jre

Guava 32.0.0-jre has two CVE's against it.
https://mvnrepository.com/artifact/com.google.guava/guava/32.0.0-jre

Note that we use Guava only in test scope so the impact of this should be very minimal.


A way using WebdriverIO?

Do you think is it possible to use WebdriverIO with Electron?
I can not find how to do, especially for manually specify how to connect chrome driver and where to find Electron's binary with WebdriverIO.
Some ideas?

Here the solution : WebdriverIO - Issue   #721
I could write documentation on on how using using Selenium and WebDriver with WebDriverIO.
But my English is poor ( I'm french :) )

Glad you found a solution :) It would be cool if you add an example of how to use webdriverio to the existing selenium and webdriver document here. It would probably help someone down the road.
You might also want to check out selenium's Python bindings. Some people prefer them because you don't need to mess with callbacks and promises. There's an example of how to use the Python bindings with nw.js here. I think the only thing you need to change to get it to work with Electron is to use Electron's chrome driver.

Create PR #2550

Awesome :+1:

Closed by https://github.com/atom/electron/pull/2550.


Allow Deployer AWS IAM Role to use the "iam:UpdateOpenIDConnectProviderThumbprint" policy

The change has been implemented.
It will appear here after the repository has been synchronized.


RFC: conventions for packaging cards

In order to make the experience of working with cards smoother for developers, we propose to bundle the Ember code together with the Rails engine code in the same repository. The Ember code will live in a client directory off the engine's root. So, the tree will look as follows:
$ tree
.
├── ...
├── client
│   ├── app
│   │   ├── ...
│   ├── index.js
│   └── package.json
├── ...
├── my_custom_card.gemspec

Our card generator will need to be modified to create the boilerplate code organized this way.
Our workflow for adding cards packaged this way can be automated. Much of this automation already exists in the custom cards Rake task.
The install task adds the custom card's entries in routes and CSS manifest, while the install_engine task adds the entries to Gemfile and client/package.json. The Ember code is installed as an addon, which means that changes to files are detected on save and visible without restarting anything in development.
Thoughts and comments welcome.

@egh's example_tahi_engine is an example of a card organized this way.
This should address #861 and #880.

sounds good to me!

it's a smoother workflow for sure. would love to merge the rails generator with the rake commands so that it's just one command.
Something for the future:  a less fragile way to add in the addons.

:+1:

@theharq
I would also like to clarify the different steps and vocabulary for building and creating plugins.
I am thinking that these are the discrete steps that we should support:

Create a new Tahi plugin. This plugin will contain a rails engine and an ember addon (in the future, it may contain an ember engine).
Suggested command to do this: rails generate tahi-plugin PLUGINNAME

Create a new card in a Tahi plugin. This is distinct from creating the Tahi plugin because a Tahi plugin can contain multiple cards.
Suggested command to do this: rails generate tahi-task TASKNAME

Install a Tahi plugin. This should do everything to make the plugin available in the Tahi repository.
Suggested command to do this: rake tahi:install-plugin PLUGINPATH | GITURL

@egh how about calling it an engine instead of a plugin?
I like the suggested commands, makes sense to me. rails generate tahi-task will also need the name of the engine (or plugin) in which to place the card.

@mikem I was thinking plugin instead of engine because a) it is a rails engine and b) it is not an ember engine. I thought calling it a plugin might distinguish it from rails/ember engines. But I don't really care, I just want to not call it a 'card', since it could contain multiple cards.
Thanks for the point about rails generate tahi-task needing another argument.

In terms of naming conventions, I just wanted to reference "Ember Engines", which may make it into Ember core soon. https://github.com/emberjs/rfcs/pull/10.
The tricky thing is that these Tahi "Add-ons/Plugins/Engines" are made of code that includes both an Ember add-on/engine and a Rails .gem/engine.  Tahi "Plugin" seems to make sense, simply because we'd be overloading 2 definitions of "Engine", with Tahi's "Engine" as a 3rd. I :+1:  Plugin
In any case, I'd just like to make the vocabulary and moving pieces clear in Docs.


[DROOLS-1043] remove references to deprecated knowledge-api module

Related PRs:
https://github.com/droolsjbpm/drools/pull/618
https://github.com/droolsjbpm/droolsjbpm-integration/pull/279


[otbn] Enable secure FIFO pointers in TLUL adapters

See https://github.com/lowRISC/opentitan/issues/9438 for context.
Signed-off-by: Michael Schaffner<EMAIL_ADDRESS>

No this came out of a review and experiments.
We would like to protect FIFO pointers on paths where instructions are being transferred.


[WIP] Bl/inflight quorum

[WIP] - Logic is done and probably works, preliminary feedback would be appreciated. Currently fixing old tests and adding new ones.
Add support for bootstrapping to the write client. While bootstrapping the initializing and leaving shards count as a single shard towards a quorum, and only if the client successfully writes to both of them.

Coverage decreased (-2.6%) to 75.419% when pulling fccadff01a88e2dbac4fcf9b7ddffec4db345c31 on bl/inflight-quorum into d917e84a8b12d36f521cb7a27dce56a893e8428a on master.

Macro comment: Looks like there are both refactoring changes (which are presumably no-op changes as far as functionality is concerned) and functionality changes in this PR. Would be nice if we can separate those into two diffs to make easier for reviewers.

Coverage decreased (-2.7%) to 75.309% when pulling e9e735c077a3b1f41db781244ef9268f039ef967 on bl/inflight-quorum into d917e84a8b12d36f521cb7a27dce56a893e8428a on master.

Hey some of the refactors are good but conflict with changes that have already been merged to master.  Can you reduce this to the minimal change to implement this?  We can do some of the refactors that you've done here as a separate independent change.
This will help us ship faster and keep this change to a minimal set we can audit and review more carefully.

Will do.


Python 3.7.7 support?

is the minimum requirement truly 3.8? I switched it to 3.7.7 and it seems to function.

I had it as 3.8 mostly because of Pathlib, but if it works with 3.7.7 I wouldn't mind changing it to that.

I now have an 3.8 so I am all set. I will comply with 3.8 requirements. No need to back test it

On Oct 23, 2020, at 4:32 PM, Alex Reed<EMAIL_ADDRESS>wrote:
I had it as 3.8 mostly because of Pathlib, but if it works with 3.7.7 I wouldn't mind changing it to that.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/areed1192/python-trading-robot/issues/7#issuecomment-715575767, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABM2NZOQNR7JTTMTRESIVWTSMHR4NANCNFSM4QUABCYQ.


[BUG] when the model is loaded and saved again, the argument names disappear (become 'args_0, args_0_1 etc)

Bug description
when the model (it is transformer based XLNet model that we are training)  is loaded and saved again, the argument names disappear (become 'args_0, args_0_1 etc) and the saved model becomes invalid. Because of that we cannot export the reloaded model using Merlin Systems, therefore cannot generate the config files to serve on Triton.
Steps/Code to reproduce bug
Please run these two gists in order to repro the issue:

https://gist.github.com/rnyak/0baa9e80cb419128379f5e91e8a30ed5
https://gist.github.com/rnyak/d222fe2eb7b2ecbf87d924a762c1f7d4#file-sbr_xlnet_load-py

Expected behavior

Environment details

Merlin version: We are using merlin-tensorflow:23.08 image with the latest main branch pulled and installed.
Platform:
Python version:
PyTorch version (GPU?):
Tensorflow version (GPU?):

One quick solution to that is to add model block line right before loading the saved model. Something like that:
_ =  mm.XLNetBlock(d_model=32, n_head=2, n_layer=2)  # replace XLNetBlock with `GPT2Block` if you use GPT2 block in model training.
loaded_model = tf.keras.models.load_model('/workspace/saved_model/')

One quick solution to that is to add model block line right before loading the saved model. Something like that:

I have the same problem, but I cannot understand this solution. Why that?


BufferedSource error with WebSockets

In WebSocketListener#onMessage(BufferedSource bufferedSource, WebSocket.PayloadType payloadType), calling bufferedSource.readUtf8() after verifying that payloadType == WebSocket.PayloadType.TEXT throws the following exception:
System.err  W  java.lang.IllegalArgumentException: byteCount < 0: -26991
    W      at okio.RealBufferedSource.read(RealBufferedSource.java:46)
    W      at com.squareup.okhttp.internal.ws.WebSocketReader$FramedMessageSource.read(WebSocketReader.java:250)
    W      at okio.RealBufferedSource.exhausted(RealBufferedSource.java:60)
    W      at com.operator.expert.service.ChatSocketConnection.onMessage(ChatSocketConnection.java:76)
    W      at com.squareup.okhttp.internal.ws.WebSocketReader.readMessage(WebSocketReader.java:112)
    W      at com.squareup.okhttp.WebSocket$ReaderRunnable.execute(WebSocket.java:283)
    W      at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
    W      at java.lang.Thread.run(Thread.java:841)

Any ideas how to prevent/fix this? It doesn't appear to happen to all socket messages.

I have identified a potential source of this error. Working on how to apply the fix since it's not a simple misread or mathematical error.


Added some small UI enhancements to the Settings and Sidebar

Add Logo icons LLM's
Added the Next.js Image component import for optimized image loading
Modified the provider item layout to include an icon next to the provider name
Used template literals to dynamically load the correct icon based on the provider name
Added proper sizing (24x24) for consistent icon display
Added dark:invert class to ensure icons are visible in dark mode (assuming they're dark by default)

Settings UI enhancement
Date & Time Display
Added a real-time clock component in the sidebar

Event Logs System
Implemented an EventLogsTab component for system monitoring
Provides a structured way to:
Track user interactions
Monitor system events
Display activity history

Br,
Stijn

thanks for this update will review and merge soon, I have checked already, but didn't get a chance to run it yet
an advice. create a branch in your fork and then do your changes in that branch, instead of working on main. it will solve lots of your problems in future 👍

Thank you for the tip Github is quite new for me that is why, i need to test how todo it... If you have the time please test thx ;-)


Jolokia Agent Tomcat 10

Hi! thanks in advance
I have jolokia in some tomcat server versions and now I would put it in tomcat 10 but I have seen that it is not compatible.it's true?
Is there a forecast when there will be a compatible version?
Thanks!

Hi
see https://github.com/rhuss/jolokia/pull/484 / https://github.com/rhuss/jolokia/pull/540

Closing as duplicate, let's continue on the other issues.

Closing as duplicate of #451.


Implement expectNext() and test cases

resolved: #43
details: Implement expectNext() and test cases
And I have one question, how we can compare error?

[x] Test case

first, sorry... plz remove Token from returning statement. I made a mistake :(

And if the code change is not about adding feature or modifying it. plz git commit --amend or squash the commit with force push

Fix wrong typo in parser_internal_test.go

And if the code change is not about adding feature or modifying it. plz git commit --amend or squash the commit with force push

Fix wrong typo in parser_internal_test.go

Ok I made it on purpose. I will do that way.


About Gradio

I hear bad things about Gradio, but is it really disconnect after about 10-ish txttoimg generations bad? Almost every time I use the WebUI, I can generate around 10 images before it error connects out, and I have to rerun the launch WebUI block and reinput everything. I think the most I was able to generate before it disconnected was 20-ish.
Edit: A4000 seems to be able to handle a lot more generations, but will still disconnect after 20-40 generations. Is there a way to avoid the disconnecting altogether?

I have the same issue, I think it has something to do with I/O, I noticed that if I disable preview images, limit interactions that would load images from the Gradio UI it delays crashes considerably.

What are you using to access your web ui? I use rathole: https://github.com/Engineer-of-Stuff/stable-diffusion-paperspace/issues/49#issuecomment-1368494704

I use whatever the default is?


ISPN-15293 Upgrade JGroups to 5.3.0.Final

https://issues.redhat.com/browse/ISPN-15293

Thanks @pruivo


Animation is lost after Gif to Avif

When I convert an animated Gif file to an Avif file, the animation is lost , only the first frame of the image is visible

Animated AVIF sequences are unsupported by libheif and therefore also libvips and sharp. Please see the upstream discussion at https://github.com/strukturag/libheif/issues/377

I hope this information helped. Please feel free to re-open with more details if further assistance is required.


add visualization for circular obstacles

This small PR provides support to publish rviz markers for obstacles of the CircularObstacle type.  It follows the form of the other obstacle publishers (point, line, and polygon).

thank you for this PR!


Occasional "Connection timed out" in distributed TensorFlow

We are occasionally observing crashes like below (training on AWS, seeing couple of crashes per day). It looks like in gRPC, recvmsg returns with "Connection timed out". The error points to this line. Default behavior in TensorFlow seems to be "wait forever", is there way to have same behavior for slow grpc connections?
@mrry
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 972, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 954, in _run_fn
    status, run_metadata)
  File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py", line 463, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.UnavailableError: {"created":"@1481750298.553120216","description":"OS Error","errno":110,"file":"external/grpc/src/core/lib/iomgr/tcp_posix.c","file_line":229,"grpc_status":14,"os_error":"Connection timed out","syscall":"recvmsg"}
     [[Node: ExponentialMovingAverage/decay_S37 = _Recv[client_terminated=false, recv_device="/job:ps/replica:0/task:0/cpu:0", send_device="/job:worker/replica:0/task:2/cpu:0", send_device_incarnation=3009888913849196393, tensor_name="edge_1032_ExponentialMovingAverage/decay", tensor_type=DT_FLOAT, _device="/job:ps/replica:0/task:0/cpu:0"]()]]
     [[Node: gradients/truediv_grad/Shape_1_S63 = _HostRecv[client_terminated=false, recv_device="/job:worker/replica:0/task:2/cpu:0", send_device="/job:ps/replica:0/task:0/cpu:0", send_device_incarnation=-6688974477600735349, tensor_name="edge_115_gradients/truediv_grad/Shape_1", tensor_type=DT_INT32, _device="/job:worker/replica:0/task:2/cpu:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bin/worker", line 231, in <module>
    tf.app.run()
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "bin/worker", line 228, in main
    run_generic(server, args, trainer, num_global_steps=num_global_steps)
  File "bin/worker", line 152, in run_generic
    trainer.process(sess)
  File "/experiment/vnc-agents/trainers/base.py", line 351, in process
    fetched = sess.run(fetches, feed_dict=feed_dict)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 717, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 915, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 965, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 985, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.UnavailableError: {"created":"@1481750298.553120216","description":"OS Error","errno":110,"file":"external/grpc/src/core/lib/iomgr/tcp_posix.c","file_line":229,"grpc_status":14,"os_error":"Connection timed out","syscall":"recvmsg"}
     [[Node: ExponentialMovingAverage/decay_S37 = _Recv[client_terminated=false, recv_device="/job:ps/replica:0/task:0/cpu:0", send_device="/job:worker/replica:0/task:2/cpu:0", send_device_incarnation=3009888913849196393, tensor_name="edge_1032_ExponentialMovingAverage/decay", tensor_type=DT_FLOAT, _device="/job:ps/replica:0/task:0/cpu:0"]()]]
     [[Node: gradients/truediv_grad/Shape_1_S63 = _HostRecv[client_terminated=false, recv_device="/job:worker/replica:0/task:2/cpu:0", send_device="/job:ps/replica:0/task:0/cpu:0", send_device_incarnation=-6688974477600735349, tensor_name="edge_115_gradients/truediv_grad/Shape_1", tensor_type=DT_INT32, _device="/job:worker/replica:0/task:2/cpu:0"]()]]

Hm, just simulated a hang locally, and the connection resumes successfully 6 hours later so it seems manual timeout is not a problem and it's probably something in grpc/recvmsg layer

Hi @yaroslavvb @mrry
I've also came across with the issue. I've implemented an online learning service using distributed TensorFlow where the training data is fed through external gRPC calls (using separate implementation from the gRPC service in TF). The code works fine when we continuous feeding training data. But when stops the feeding for some period and resumes again. The session.run operation sometimes got stuck. The training service is hosted on Azure. TF version is 0.12RC1 and OS version Ubuntu16.04.
Thanks!

Traceback (most recent call last):
  File "/home/XXX/OnlineLearningService/python/service/dispatcher.py", line 75, in update_explr_rate
    current_training_step = self.session.run(self.global_step)
  File "/home/XXX/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 717, in run
    run_metadata_ptr)
  File "/home/XXX/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 915, in _run
    feed_dict_string, options, run_metadata)
  File "/home/XXX/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _do_run
    target_list, options, run_metadata)
  File "/home/XXX/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 985, in _do_call
    raise type(e)(node_def, op, message)
UnavailableError: {"created":"@1481767018.578522717","description":"OS Error","errno":110,"file":"external/grpc/src/core/lib/iomgr/tcp_posix.c","file_line":229,"grpc_status":14,"os_error":"Connection timed out","syscall":"recvmsg"}

So my reading of this error is that recvmsg call at this line of grpc code returned with ETIMEDOUT error
GPR_TIMER_BEGIN("recvmsg", 0);
  do {
    read_bytes = recvmsg(tcp->fd, &msg, 0);
  } while (read_bytes < 0 && errno == EINTR);
  GPR_TIMER_END("recvmsg", read_bytes >= 0);

There's a way to set timeout for a socket using setsockopt, but I couldn't find TensorFlow would be doing that by default, and I tested that it can successfully hang for 6 hours. So I guess the question is -- why would recvmsg fail with a timeout error? Perhaps that the behavior when connection is dropped/interrupted?


Model fields with "+", "-" should not ignore them

A # comment {
  {
  b,
  # comment
  b?,
  -b,
  -b?: i,
  -b?: i,
}

Right now for some reason, we skip -, + in fields name. Need to parse it

Do we really need to parse them?
First about -
Because - in field treats as skip: true property in node. Maybe it's much better to just handle this output in printer like that
if (skip) {
  return `-${field}`
}

About + sign,I ban this type of fields in lexer because it kind of weird thing to do. I don't know cases for now where it can be useful have + sign in your fields

Close due to clarification


Unable to load the saved_model

I am running the model for countries_s1 model but getting the following error:
Executing python code/model/trainer.py --base_output_dir output/countries_s1/ --path_length 2 --hidden_size 25 --embedding_size 25     --batch_size 256 --beta 0.05 --Lambda 0.05 --use_entity_embeddings 1     --train_entity_embeddings 1 --train_relation_embeddings 1     --data_input_dir datasets/data_preprocessed/countries_s1/ --vocab_dir datasets/data_preprocessed/countries_s1/vocab --model_load_dir saved_models/countries_s1/model.ckpt --load_model 1 --total_iterations 1000 --nell_evaluation 0
HEY
Arguments:
                         LSTM_layers : 1
                              Lambda : 0.05
                     base_output_dir : output/countries_s1/
                          batch_size : 256
                                beta : 0.05
                        create_vocab : 0
                      data_input_dir : datasets/data_preprocessed/countries_s1/
                      embedding_size : 25
                          eval_every : 100
                               gamma : 1
                      grad_clip_norm : 5
                         hidden_size : 25
                          input_file : train.txt
                         input_files : ['datasets/data_preprocessed/countries_s1//train.txt']
                        l2_reg_const : 0.01
                       learning_rate : 0.001
                          load_model : True
                             log_dir : ./logs/
                       log_file_name : output/countries_s1//3f71_2_0.05_100_0.05/log.txt
                     max_num_actions : 200
                           model_dir : output/countries_s1//3f71_2_0.05_100_0.05/model/
                      model_load_dir : saved_models/countries_s1/model.ckpt
                     negative_reward : 0
                     nell_evaluation : 0
                        num_rollouts : 20
                          output_dir : output/countries_s1//3f71_2_0.05_100_0.05
                         output_file :
                         path_length : 2
                    path_logger_file : output/countries_s1//3f71_2_0.05_100_0.05
                                pool : max
                     positive_reward : 1.0
        pretrained_embeddings_action :
        pretrained_embeddings_entity :
                       test_rollouts : 100
                    total_iterations : 1000
             train_entity_embeddings : True
           train_relation_embeddings : True
               use_entity_embeddings : True
                           vocab_dir : datasets/data_preprocessed/countries_s1/vocab
INFO:root:reading vocab files...
01/28/2019 01:56:20 AM: [ reading vocab files... ]
INFO:root:Reading mid to name map
01/28/2019 01:56:20 AM: [ Reading mid to name map ]
INFO:root:Done..
01/28/2019 01:56:20 AM: [ Done.. ]
INFO:root:Total number of entities 273
01/28/2019 01:56:20 AM: [ Total number of entities 273 ]
INFO:root:Total number of relations 6
01/28/2019 01:56:20 AM: [ Total number of relations 6 ]
INFO:root:Skipping training
01/28/2019 01:56:20 AM: [ Skipping training ]
INFO:root:Loading model from saved_models/countries_s1/model.ckpt
01/28/2019 01:56:20 AM: [ Loading model from saved_models/countries_s1/model.ckpt ]
Reading vocab...
batcher loaded
KG constructed
Reading vocab...
Contains full graph
batcher loaded
KG constructed
Reading vocab...
Contains full graph
batcher loaded
KG constructed
2019-01-28 01:56:21.112771: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2019-01-28 01:56:21.112794: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2019-01-28 01:56:21.112801: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2019-01-28 01:56:21.112806: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2019-01-28 01:56:21.112812: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2019-01-28 01:56:21.988648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.582
pciBusID 0000:06:00.0
Total memory: 10.91GiB
Free memory: 10.75GiB
2019-01-28 01:56:21.988740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2019-01-28 01:56:21.988757: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y
2019-01-28 01:56:21.988799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0)
INFO:root:Creating TF graph...
01/28/2019 01:56:22 AM: [ Creating TF graph... ]
INFO:root:TF Graph creation done..
01/28/2019 01:56:23 AM: [ TF Graph creation done.. ]
INFO:tensorflow:Restoring parameters from saved_models/countries_s1/model.ckpt
INFO:tensorflow:Restoring parameters from saved_models/countries_s1/model.ckpt
01/28/2019 01:56:23 AM: [ Restoring parameters from saved_models/countries_s1/model.ckpt ]
2019-01-28 01:56:23.830867: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
2019-01-28 01:56:23.846931: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847248: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847249: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847266: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847290: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847248: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847342: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847358: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847437: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847529: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847785: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847753: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847603: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847794: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847881: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.847816: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.848095: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.848317: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.848375: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.854893: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.854890: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.854966: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.854933: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.855353: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.855395: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam not found in checkpoint
2019-01-28 01:56:23.855675: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.855750: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.855791: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.855820: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.855785: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.856273: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.856468: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.856486: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.856592: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.856623: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
2019-01-28 01:56:23.856677: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
Traceback (most recent call last):
  File "code/model/trainer.py", line 573, in <module>
    trainer.initialize(restore=save_path, sess=sess)
  File "code/model/trainer.py", line 143, in initialize
    return  self.model_saver.restore(sess, restore)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1560, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1124, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
    options, run_metadata)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
         [[Node: save/RestoreV2_17/_3 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_80_save/RestoreV2_17", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

Caused by op u'save/RestoreV2_9', defined at:
  File "code/model/trainer.py", line 573, in <module>
    trainer.initialize(restore=save_path, sess=sess)
  File "code/model/trainer.py", line 137, in initialize
    self.model_saver = tf.train.Saver(max_to_keep=2)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1140, in __init__
    self.build()
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1172, in build
    filename=self._filename)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 688, in build
    restore_sequentially, reshape)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 407, in _AddRestoreOps
    tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 247, in restore_op
    [spec.tensor.dtype])[0])
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 663, in restore_v2
    dtypes=dtypes, name=name)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/ext_user2/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

NotFoundError (see above for traceback): Key entity_lookup_table/entity_lookup_table/Adam_1 not found in checkpoint
         [[Node: save/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_9/tensor_names, save/RestoreV2_9/shape_and_slices)]]
         [[Node: save/RestoreV2_17/_3 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_80_save/RestoreV2_17", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

It looks like the model was training with the train_entity_embeddings parameter set as False. Could you change that in the config and rerun?

@kunal017 Have you solved this problem? I also encountered
NotFoundError (see above for traceback): Key policy_steps_unroll/multi_rnn_cell/cell_0/lstm_cell/bias not found in checkpoint 	 [[Node: save/RestoreV2_18 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_18/tensor_names, save/RestoreV2_18/shape_and_slices)]]
@shehzaadzd Can you help me?

Can you try training from scratch with the same parameters?

Yes, retrain and saved model can be successfully loaded


⚠ Guya.moe Proxied has degraded performance

In 7e7a081, Guya.moe Proxied (https://ice.guya.moe/) experienced degraded performance:

HTTP code: 200
Response time: 11961 ms

Resolved: Guya.moe Proxied performance has improved in 2868d5c after 1 hour, 18 minutes.