ParNCMesh I/O and new NC mesh format [par-ncmesh-saving-dev]

There are currently several problems with our nonconforming mesh format:

ParNCMesh cannot be saved to a file, because the MFEM mesh v1.1 format does not support storing element rank assignments (#1332).
v1.1 does not support storing root element orientation (NCMesh::root_state). This could be causing bugs like #1326, if the orientation is lost.
The v1.1 format was designed as a minimal extension to the conforming format v1.0, which in hindsight was not fortunate. The format does not mirror well the NCMesh internal data structure: elements and vertices need to be renumbered when saving/loading, and empty slots in the element/node arrays are not supported. As a result, the NCMesh is not identical when saved and loaded again.
The parallel v1.2 format completely ignores NC meshes and is in fact another branch off v1.0. A potential v1.3 for parallel nonconforming meshes does not make sense.

This PR proposes the following:

A new MFEM NC mesh v1.0 format is introduced, independent of the conforming formats MFEM mesh v1.0 and v1.2.
The v1.1 format is deprecated, although it can still be loaded, for backward compatibility.
The new nonconforming format better mirrors the actual data structure and removes known inconsistencies. It works both in serial and in parallel, as the only parallel information is the optional rank section and the element rank member. Everything else in ParNCMesh is calculated from the ghost layer.
The new format is still human readable and editable and is similar in style to the conforming format.
Example 6p now illustrates how checkpoints can be saved and later loaded to restart the computation.

Example of the new format
The following file was obtained by converting a 1x1 inline quad mesh to an NC mesh and refining it once:
MFEM NC mesh v1.0

# NCMesh supported geometry types:
# TRIANGLE    = 2
# SQUARE      = 3
# TETRAHEDRON = 4
# CUBE        = 5
# PRISM       = 6

dimension
2

# optional rank for parallel files, defaults to 0
rank
0

# rank attr geom ref_type nodes/children
elements
6
0 1 3 3 1 2 3 4     # this is a non-leaf (refined) element
0 1 3 0 0 4 8 7
0 1 3 0 4 1 5 8
0 1 3 0 8 5 3 6
0 1 3 0 7 8 6 2
0 1 -1              # empty slot (freed ID in NCMesh::elements)

# attr geom nodes
boundary
8
1 1 0 4
4 1 7 0
1 1 4 1
2 1 1 5
2 1 5 3
3 1 3 6
3 1 6 2
4 1 2 7

# vert_id p1 p2
vertex_parents
5
4 0 1
5 1 3
6 2 3
7 0 2
8 4 6

# root element orientation
root_state
1
0

# top-level node coordinates
coordinates
4
2
0 0
1 0
0 1
1 1

mfem_mesh_end

Notes:

Both refined (ref_type != 0) and leaf elements (ref_type == 0) are listed in the elements section. This replaces the problematic coarse_elements section of v1.1. Refined elements reference child element IDs instead of node IDs, as in NCMesh. Element indices correspond 1:1 to the array NCMesh::elements. Empty entries can be represented by setting geom to -1.
boundary and vertex_parents sections are the same as in v1.1. Vertex parents can be listed in any order and the IDs do not have to be contiguous. The IDs correspond 1:1 to NCMesh vertex nodes (i.e., not Mesh::vertices). Edge nodes are not stored and are assigned any of the free IDs on load.
root_state stores the first N entries of NCMesh::root_state, the remaining entries (if N < M) are assumed to be zero. Here M is the number of root elements, automatically detected in the elements section (the first M elements must not have a parent).
The coordinates section replaces vertices of v1.1 and only lists the coordinates of the top-level nodes (coordinates of other nodes are calculated from the parent relations in NCMesh). Only the first N node IDs should be top-level nodes (these have no parents and are thus not listed in vertex_parents). The section can be longer but we check that it covers all top-level nodes. When loading vertices from v1.1, we now trim the array to only cover the top-level nodes.
If the coordinates section is missing, we expect to find a nodes section instead of it. The Nodes are loaded the same way as in v1.0 (it's just not a subsection of coordinates/vertices).
The sections rank, boundary, vertex_parents and root_state are all optional (their order must be kept though, if present). Exacly one of coordinates or nodes must be present.
The file ends with mfem_mesh_end as in v1.2, which is not strictly necessary, but could be useful.

More implementation notes:

v1.1 can still be loaded as a Mesh, but not as a ParMesh. Although technically possible (all ranks would load the same file and then Rebalance could be called), it would probably be confusing.
Changes in ParMesh and ParNCMesh are in fact minimal. Everything is handled by the serial NCMesh and ParNCMesh just calculates all its structures from the ghost layer.
All parallel files (like mesh.000000) are valid serial NC meshes that can be displayed individually in GLVis. The corresponding ParGridFunction files are now also compatible.

UPDATE

The serial code now knows the necessary minimum about ghost elements (those with element.rank != MyRank), ghost vertices, ghost edges and faces, which allows it to skip the ghosts when creating the Mesh representation seen by the rest of MFEM.
The member ParNCMesh::MyRank is now NCMesh::MyRank and it gets loaded from the rank section. When loading a mesh in parallel, the rank is checked to equal the actual MPI rank.
NCMesh::UpdateVertices now works the same in both serial and parallel and enumerates vertices in specific order to keep compatibility with conforming meshes, and between serial and parallel NC meshes (see code for details).
Similarly, NCMesh::OnMeshUpdated now handles ghost edges and faces, so that they are indexed the same way in serial and in parallel.
As a result, parallel solutions on NC meshes should now display in serial GLVis, both complete and as individual parts.
Possible TODO: processor boundary faces are not output anywhere explicitly as in ParMesh::Print, so 3D parts are displayed incomplete, I'm not sure how to handle this (I think GLVis should generate this information itself, but that's not how it works now).

TODO

[x] Compatibility with partial grid functions?
[x] Fix on the fly NC initialization (polar-nc)
[x] Bug? Empty coarse_elements in legacy format required?
[x] Mesh::SetCurvature must remove NCMesh::coordinates
[x] ParNCMesh::GetFaceNeighbors crashes on curved meshes (coordinates.Size() == 0)
[x] ExchangeFaceNbrData on curved meshes: "Nodes are not ParGridFunction!"
[x] Convert v1.1 meshes in the repo.
[x] Fix loading curved v1.1 meshes
[x] Crash when restarting ex6p in NC tet mesh (escher)
[ ] Document the new format.
[x] Update ex6p description.
[x] Update CHANGELOG.

PR
Author
Editor
Reviewers
Assignment
Approval
Merge

#1392
@jakubcerveny
@v-dobrev
@mlstowell + @tzanio
10/06/20
12/29/20
⌛due 01/05/21

There's some more TODO items but people may start looking at this and comment on the format...

I was looking into the problem of making partial ParGridFunctions compatible with partial NC mesh files, but it's more tricky than I thought. I will have to modify both NCMesh and ParNCMesh to make sure the elements and vertices are enumerated exactly the same way in parallel and serial.
@tzanio can we postpone the reviews please?

@tzanio can we postpone the reviews please?

Done. Mark as ready-for-review when it is ready again.

Hi guys,
first off, thanks for the effort! I have two questions on this branch.

Is it possible to reset the internal tree data structure for an [Par]NCMesh?
To elaborate on what I mean, I have written a tool on top of MFEM, which is somewhat similar to the Shaper miniapp, to generate a (nonconforming) mesh from voxel image data with the property of a minimum number of elements. I want to fix these elements to be the new roots for my mesh.

Is there an alternative way or a hotfix to store and load nodal information (in contrast to the not yet fully functional [Par]GridFunction), which can be made available as a coefficient for the integrator classes?
My imaging data also contains the spatially varying coefficients for my PDE, basically giving me information for a spacially varying diffusion coefficient field, therefore I also must save associated information. I my current implementation I realize this by storing a ParGridFunction and hand it to the DiffusionIntegrator via a VectorGridFunctionCoefficient.

Best regards,
Dennis

Hi Dennis,

A function to flatten the NCMesh does not exist, but could be written. Just create a new NCMesh::elements that contains the leafs only, and resize (and zero-out) NCMesh::root_state. Oh and you may need to renumber the element references in NCMesh::Face::elem. I'm not sure if this actually pays off because in a uniformly refined hex mesh the number of coarse elements is ~14% (1/7) of the number of leafs.

I have now tested ex6p and the solution, when the NC mesh is saved with ParPrint, is broken because the ordering of elements is different. I still need to look into this. In the meantime I suggest you use the master branch, sorry.

Jakub

Hi Jakub,
thanks for the quick answer! And no reason to sorry. :)
The flattening procedure sounds really doable. I will take a look into this. To make the idea in my first point clear, it is not about saving memory, but about having a as coarse as possible mesh with fixed material interfaces. I want these to be kept in their place when running simulations with AMR (since derefinement could mess up the resolution of these material interfaces).
Best regards,
Dennis

Hi again,
I realized an issue related to element attributes with all available file formats. Since this PR redesigns the NC mesh format, this may be relevant.
All formats discard the attribute assignment for lower-dimensional entities, i.e. the attributes for points, edges and faces in a 3d mesh (point/point+edge attributes in 1d/2d respectively) are lost when saving and loading a mesh. This is relevant in simulations of models with multiple materials, in which material interfaces may require a different treatment, depending on non-local properties determined in preprocessing steps.
My proposal here would be to include optional sections (e.g. face_attributes, ...) for the attributes of lower dimensional entities. This way the format would stay compatible and if attributes for lower dimensional entities are not used, the saving them could be skipped.
Best regards,
Dennis

Hi Dennis,
I'm not sure I understand. In MFEM you can have element attributes and boundary element attributes (boundary elements, if present, are dim-1 dimensional). We do save these. If you wanted vertex/edge/face attributes, these would have to be supported in MFEM in the first place. Can you describe your use case?
Jakub

Hi Jakub,
thanks for the quick response. I simulate electrophysiologcal models of heart tissues. These tissues are usually either embedded into other biological tissues or in some fluid. Face attributes are useful for storing information about "internal" material interfaces, e.g. for the boundary layers between different parts in the heart or between the heart and its surrounding. I need the face attributes frequently in all steps of my pipeline, spanning preprocessing, the simulation itself (because this information becomes handy in the coupling scheme I use) and postprocessing.
For me it would be a nice to have, but I can also get away by storing this face information separately and managing it manually. I just wanted to point towards possible issue. If it is intended to not store such information, then it is (at least for me) also perfectly fine.
Regarding the support for face attributes, if I understand the implementation correctly, then I can access face attributes by first getting the face instance via
https://github.com/mfem/mfem/blob/a25138e61179753291e0a128a94fae4e540e2bf9/mesh/mesh.hpp#L804
and then there is the following method
https://github.com/mfem/mfem/blob/62117e96dd3d9f64dc5d679a1b029cc802effa2a/mesh/element.hpp#L54-L58
where I tried to be consistent with the current implementation, i.e.
https://github.com/mfem/mfem/blob/a25138e61179753291e0a128a94fae4e540e2bf9/mesh/mesh.hpp#L1021-L1028
Best regards,
Dennis

You are right, I forgot we do hold faces explicitly in Mesh::faces. I'm not sure this is a place for users to store attributes but it is true Mesh::GetFaceTransformation works with them. Is this the intended design @v-dobrev ?
My personal guess would be that on material interfaces you should be able to define "boundary" elements inside the mesh and then see their attributes in face integrators etc., but I may be wrong.

The idea for what we call "boundary elements" is that some of them can be actually internal faces describing interfaces. How these are actually used is up to the application, e.g. you probably do not want to apply boundary conditions there.
Does merging the interior (interface) faces with the real boundary faces make sense in your application?
One issue with maintaining attributes for all internal faces is that when you refine the mesh you create new faces that are not a refinement of existing faces, so there is no natural attribute assigned to those. That's one reason why to me it only makes sense to save attribute data for subsets of the faces and we just merge those with the real boundary elements.
What we do not currently support is storing attributes for lower-dimensional mesh entities like edges (in 3D) and vertices (in 2D and 3D). Is that something you need too? One option that has come up is to store them as "boundary elements" too -- that way the mesh format does not need to change.
One other thing, we currently do not copy the attributes from the boundary elements to their respective face elements (which are stored as separate objects in the Mesh class). I'm not sure if this will be useful to have. That's why the face attributes are not really used at this point, even though we copy the attributes in some places like Mesh::GetFaceTransformation.

Thanks for the elaborate explanation and your notes! That actually helps me to clear up several implementation-related uncertainties I had. I also totally agree on the point, that it really does not make sense to store attributes (d-2)-dimensional entities for the mentioned reason. Luckily I do not need attributes on these entities. Being able to reuse boundary elements for the internal boundaries should already be more than enough for my application.
Thanks!

Hi Guys, what's the status of this new mesh? If we use this new mesh, can we save the adaptive mesh and solution, and then restart from there?
I start to do some multiscale simulations with my solver. The debug has been really challenging now for those hard cases without restart capability.

@tangqi This is is not ready yet, sorry, I had to switch to other things. I will return to this ASAP.

@tangqi I think this should already work for your use case (you may need to merge this branch into your own though). I modifed ex1p and added examples/parprint-test to demonstrate loading/saving of parallel NC solutions. Whatever you compute with ex1p is saved with the new ParPrint and loaded in parprint-test, where you can also refine and rebalance the solution, to check that it was loaded correctly and still works when modified.
What doesn't work yet is visualizing the new format in GLVis, in particular visualizing the partial files of a parallel solution, I need to make more changes to support that.

Awesome! Thanks @jakubcerveny  I will run some tests on my branch then.

Merged ncmesh-serial-ghosts-dev into this branch. Serial code now knows about ghost elements and the algorithm to assign vertex, edge and face indices has been unified. Serial code (like GLVis) can now load parallel partial meshes and GridFunctions.

I'll fix the crash when loading parallel NC tet meshes next week, but otherwise this is finished.

Hi Jakub,
I have two question related to the free id's. In your example you have a mesh file using the new format, generated by utilizing a single 2D element and refining it once.

I would expect to see 5 elements (root + four children), so where does the "free" element come from? Or is it manually added for demonstration purposes?
I also observed that the boundary elements are numbered from 1 to 4. I can observe the same behavior in my remeshing tools (which are implemented on top of MFEM) and that changes in the boundary attribute assignment are not properly propagated to the ncmesh writer. Probably more related to  #1610 and #1572 , but I just want to ask whether the former behavior is intended.

Hi @termi-official, sorry for the late answer (I have been a bit ill the past few days).

The empty slot in the example is just for illustration (I added it manually). Empty slots only occur if some elements have been derefined and their slots have not been reused yet.

You are right that changes in boundary attributes are not propagated from the Mesh to the NCMesh. This is an issue that is not really related to this PR, please feel free to add code that does this (possibly along with other attribute related changes we discussed some time ago) and we can make it another PR.

Hi @jakubcerveny ,
no problem, I hope you are well again! And thanks for the clarification.
Regarding the boundary attributes I just wanted to double check if this is the same issue as we discussed already or whether this could be something else, because the default attribute assignment looked a bit strange to me. But let us take look into this in pr 1610 after this here is merged.
I am currently testing this branch against my implementation and for now the new IO on non-conforming meshes works like a charm. However, I still need at least a week or two to fully integrate and test this properly. I want to note that the last time I integrated this branch (few months ago already) load balancing broke the attribute assignment. For now I was not able to reproduce this with ex6p and ex21p, so I will update you on this next week.
Best regards,
Dennis

Hi, @jakubcerveny ,
I would like to suggest a small change to ex6p.  Add a command line option to specify the value of max_dofs.  This would enable us to test AMR after a restart.  Presumably it will work fine but this will make the restart option a little more meaningful in this example.
Best wishes,
Mark

The joule sample run which uses the new cylinder-tet.mesh fails due to a triangular face with an unsupported orientation.   I checked master and this seems to be a new issue in this PR.
Also, the sample run using the cylinder-hex-q2.mesh appears to go unstable but, unfortunately, this also happen in master so this may have been a previously unnoticed problem.

The serial unit tests fail however this PR is not the most likely culprit.  Perhaps merging the latest master branch into this one will resolve this.

I ran the restart feature of ex6p, with a couple extra AMR iterations, though valgrind and didn't see any noteworthy output.

@mlstowell, I think the problem with joule is not related to this PR -- I am actually able to get this error in master too: #1874.
The reason this is visible now is that in master, cylinder-tet.mesh is a v1.1 mesh without a vertex_parents section, so the mesh is actually loaded as conforming. But if you do mesh->EnsureNCMesh(true) (force simplices to be NC), you get the same error. I created the above issue to address it.
In this PR, we should maybe keep cylinder-tet.mesh a conforming (v1.0) mesh instead of converting it to the new NC format?

I merged latest master and when I do make pdebug; make test I don't get any errors...

@mlstowell, I think the problem with joule is not related to this PR -- I am actually able to get this error in master too: #1874.
The reason this is visible now is that in master, cylinder-tet.mesh is a v1.1 mesh without a vertex_parents section, so the mesh is actually loaded as conforming. But if you do mesh->EnsureNCMesh(true) (force simplices to be NC), you get the same error. I created the above issue to address it.
In this PR, we should maybe keep cylinder-tet.mesh a conforming (v1.0) mesh instead of converting it to the new NC format?

Yes, I think that's best.  I just double-checked that the original mesh works in this new branch if we just change the version in the header from v1.1 to v1.0.  Should we keep the new mesh?  If it was non-trivial to create it, I think we should keep it somewhere.  I'll leave that up to you, @jakubcerveny.

Saving "empty slots" is kind of strange -- this is a byproduct of how we store things internally. I guess you need it for recovering the internal state which influences future id assignments. That, however, may not be sufficient since we don't know the order in which the "empty slots" are listed in the free_element_ids array. Could this be a problem?
We can probably introduce "empty slot removal" step after refinement/derefinement to fill-in the empty slots with ids from the end. This, however, will require updating element ids which may not be that easy.

I think the problem is that we are trying to come up with a human readable format that is also a serialization format. The empty slots are a compromise. Normally they are pretty rare (typically they are immediately reused in dynamic AMR) and if the file is created manually, the user does not have to care or even know about them.
The alternative is an explicit element ID on each line in the elements section, but that's mostly redundant and it would be another deviation from the v1.0 style. I would also like to avoid any renumbering of the IDs.

Hi @termi-official,
Thanks for the review. Load balancing never cared about element attributes and I don't believe this actually works in master. Although this is a valid use case, I never had it in mind when writing the Par/NCMesh code (the assumption was that the whole refinement tree shares the attribute of its root element). Handling NC element attributes correctly will require a project of its own. Do you think you could summarize in an issue all the things that don't work regarding attributes in your use cases (I started losing track of them), and we'll make a PR to fix this.
Converting a conforming ParMesh to an NC ParMesh is not possible. One needs to convert while the mesh is still serial. I added a check in EnsureNCMesh -- good catch.
A parallel NC mesh needs to be created from a single serial Mesh that is assumed to be coarse enough to fit on all MPI ranks. This is of course a strong assumption but it greatly simplifies the parallel NC implementation, so it was accepted at the very beginning.
As for NCMesh::Trim, it would be possible to release unused blocks at the end of BlockArray. I added a TODO comment for the future.
Thanks again for testing this PR thoroughly.

Hi Jakub,
glad to read that this one was helpful for you and thanks for the comments. I also want to note that my mentioned issues on the attribute management on NC meshes are currently not too important for me, because I have duck taped some patches which work for me in my project.
I have some more questions regarding restarts, which I forgot in my previous post.
Since this PR basically unlocks AMR simulations with restarts, do we have DataCollections supporing this feature? I skipped through all subclasses and I think only the VisItDataCollection may be capable of this, correct? If so, are there any future plans to include a HDF5 (or similar) based format? If the VisItDataCollection is also uncapable of handling this, I have some very basic MFEM-specific DataCollection, which I could factor out of my project and contribute. My implementation is not really efficient, but it works.
 For example in my current project I use the result of a first FEM simulation as the input for a second FEM simulation.

Hi Dennis,
I am not too familiar with DataCollections, but on a quick look it seems DataCollection::SaveMesh is able to use pmesh->ParPrint (if format == PARALLEL_FORMAT), so my guess would be that the parallel NC format could just work with DataCollections. Did you try this?

Hi Jakub,
output with the DataCollection works fine, but the Load method is "not implemented" (i.e. throws an exception), which makes sense, because we are missing meta information at this point. That is why I subclassed the DataCollection to store some additional metainformation, which works.
Going through the Subclasses the SaveMesh is never used and Save is usually overwritten. The ParaViewDataCollection works fine for just outputting data, but loading will not work, because the underlying refinement tree is missing. For the VisItDataCollection loading and storing should work fine, but the class is at least missing the cycle information necessary to load the collection (i.e. the only way I to reconstruct the cycle number seems to be parsing the written folder names, if I have not missed something obvious here). This should be rather easy to fix. For the remaining DataCollections I am not even sure if the output works properly.

OK, sounds like a good topic for another PR, if you feel like contributing your improvements.

Quoting from https://github.com/mfem/mfem/pull/1392#issuecomment-725731237:

Saving "empty slots" is kind of strange -- this is a byproduct of how we store things internally. I guess you need it for recovering the internal state which influences future id assignments. That, however, may not be sufficient since we don't know the order in which the "empty slots" are listed in the free_element_ids array. Could this be a problem?

@jakubcerveny, I don't think you answered my question above. To clarify a bit: since we are storing the "empty slots" by marking them, we do lose their order as stored internally in free_element_ids; that in turn means we are NOT recovering the internal state 100%, hence my question. Isn't such 100% recovery the reason to want to store the "empty slots" in the first place?

@v-dobrev, the main reason to store empty slots is to preserve exact element IDs, which occasionally requires skipping unused indices in NCMesh::elements. You are right that the ordering of free_element_ids is not preserved (it will be sorted on load), so the recovery is just 99.9% :-) But future element ID assignment is not really important, it's the current set of active IDs that matters.

@v-dobrev, the main reason to store empty slots is to preserve exact element IDs, which occasionally requires skipping unused indices in NCMesh::elements. You are right that the ordering of free_element_ids is not preserved (it will be sorted on load), so the recovery is just 99.9% :-) But future element ID assignment is not really important, it's the current set of active IDs that matters.

Okay, good. I just wanted to double check if the ordering is important.

Merged in next for testing...

Re-merged in next.

The problems in autotest seem to be caused by changes in edge/face orientation, especially the tesla miniapp has flipped signs in the linear system. Still debugging...

The problems in autotest seem to be caused by changes in edge/face orientation, especially the tesla miniapp has flipped signs in the linear system. Still debugging...

I was able to reproduce this more simply in the RHS of ex3p on ball-nurbs.mesh on 2 CPUs. Should be easier to find now.

I was able to reproduce this more simply in the RHS of ex3p on ball-nurbs.mesh on 2 CPUs. Should be easier to find now.

So it's the ldof_signs, again. In this branch, the vertices of an NC mesh are ordered in such a way that if (i,j) is an edge and i<j on some processor, the corresponding vertices on another processor are ordered the same way. This means that the ldof_sign array is all positive, which I thought is a good thing. But apparently somewhere in the code the shared edges are broken because of this (for Hcurl and Hdiv spaces).

So it's the ldof_signs, again. In this branch, the vertices of an NC mesh are ordered in such a way that if (i,j) is an edge and i<j on some processor, the corresponding vertices on another processor are ordered the same way. This means that the ldof_sign array is all positive, which I thought is a good thing. But apparently somewhere in the code the shared edges are broken because of this (for Hcurl and Hdiv spaces).

It was a pain but I think I found the problem. The ldof_signs was a false alarm. It is true there is a difference now between the new NC mesh and the master branch in edge orientation, due to the new globally consistent ordering of the vertices. This causes some edge DOFs in H(curl) spaces to be inverted and as a result there are sign flips in the linear system and the RHS. But this is perfectly OK, IMO.
The problem is actually with HypreAMS when it's calling ParMesh::GetVertex(i) on NC curved meshes. There is a warning in Mesh::GetVertex that it shouldn't be called for high-order meshes. In this branch the vertices are actually uninitialized (I would have never found this without valgrind), which I'll try to fix, but it's partly an old problem. I think we should make sure GetVertex always works, even on curved meshes.

It was a pain but I think I found the problem. The ldof_signs was a false alarm. It is true there is a difference now between the new NC mesh and the master branch in edge orientation, due to the new globally consistent ordering of the vertices. This causes some edge DOFs in H(curl) spaces to be inverted and as a result there are sign flips in the linear system and the RHS. But this is perfectly OK, IMO.
The problem is actually with HypreAMS when it's calling ParMesh::GetVertex(i) on NC curved meshes. There is a warning in Mesh::GetVertex that it shouldn't be called for high-order meshes. In this branch the vertices are actually uninitialized (I would have never found this without valgrind), which I'll try to fix, but it's partly an old problem. I think we should make sure GetVertex always works, even on curved meshes.

I made sure vertices are set to valid coordinates even though Nodes are present, which is the previous behavior. However, I think the proper solution is to set vertices to zero size if Nodes are active, and modify GetVertex to be able to handle high-order meshes.

I made sure vertices are set to valid coordinates even though Nodes are present, which is the previous behavior. However, I think the proper solution is to set vertices to zero size if Nodes are active, and modify GetVertex to be able to handle high-order meshes.

Re-merged in next for testing ...

Re-merged in next for testing ...

Thanks @jakubcerveny. I will take a closer look at HypreAMS and this issue next week.

Thanks @jakubcerveny. I will take a closer look at HypreAMS and this issue next week.

I was looking at the remaining differences in AMR runs in autotest. They are caused by the new ordering of vertices in NCMesh, which I tested by reverting to the old ordering. Then, in a serial run of ex6, I reproduce exactly the autotest log. The parallel runs of ex6p and volta are not exactly the same as the autotest, maybe I'm not running them the same or there are machine differences. But when I compare par-ncmesh-saving-dev and master on my machine, I get an exact match (mpirun -n 4 ex6p):

The old ordering must of course normally be kept #ifdefed out.

I was looking at the remaining differences in AMR runs in autotest. They are caused by the new ordering of vertices in NCMesh, which I tested by reverting to the old ordering. Then, in a serial run of ex6, I reproduce exactly the autotest log. The parallel runs of ex6p and volta are not exactly the same as the autotest, maybe I'm not running them the same or there are machine differences. But when I compare par-ncmesh-saving-dev and master on my machine, I get an exact match (mpirun -n 4 ex6p):

The old ordering must of course normally be kept #ifdefed out.

Re-merged in next for testing ...

Re-merged in next for testing ...

There was no mention of the "new vertex ordering" in the above descriptions -- I thought this was just adding/changing file i/o. What's the motivation for changing the vertex ordering?

There was no mention of the "new vertex ordering" in the above descriptions -- I thought this was just adding/changing file i/o. What's the motivation for changing the vertex ordering?

I agree it should have been mentioned more explicitly (it's hidden in the "UPDATE" section). There is however a huge comment in the code in NCMesh::UpdateVertices, which explains the reasoning:

In master, the serial NCMesh and parallel ParNCMesh are using a different ordering of vertices. This is unnecessary and inconsistent but more importantly, it prevents the serial code from loading parallel partial GridFunctions e.g. for visualization. This is fixed now in this branch.
In master, if you load a GridFunction (even in serial) and then change the mesh to NCMesh, the GridFunction will break because of the difference of vertex ordering between conforming/nonconforming meshes. This could be considered a bug and this branch fixes that too.
The new ordering is globally consistent which also makes edge orientations consistent in parallel (I want to use that in #1274).

I agree it should have been mentioned more explicitly (it's hidden in the "UPDATE" section). There is however a huge comment in the code in NCMesh::UpdateVertices, which explains the reasoning:

In master, the serial NCMesh and parallel ParNCMesh are using a different ordering of vertices. This is unnecessary and inconsistent but more importantly, it prevents the serial code from loading parallel partial GridFunctions e.g. for visualization. This is fixed now in this branch.
In master, if you load a GridFunction (even in serial) and then change the mesh to NCMesh, the GridFunction will break because of the difference of vertex ordering between conforming/nonconforming meshes. This could be considered a bug and this branch fixes that too.
The new ordering is globally consistent which also makes edge orientations consistent in parallel (I want to use that in #1274).

Re-merged in next for testing ...

Re-merged in next for testing ...

One more thing: due to the new vertex ordering, it seems like AMR Mesh + GridFunction generated and saved by the current master branch and previous MFEM versions will no longer be readable by this branch. If I understand this correctly, the mesh itself will be fine, however, the GridFunction data will be permuted and with flipped signs in some cases. Is that correct?
If this is the case, this deserves to be mentioned in the CHANGELOG in a separate bullet.
This will also cause issues with the MFEM plugin in VisIt which is based on an older MFEM version.
The best solution will be to support reading correctly the previous default AMR mesh format and respective GridFunctions. How hard will this be to implement?

One more thing: due to the new vertex ordering, it seems like AMR Mesh + GridFunction generated and saved by the current master branch and previous MFEM versions will no longer be readable by this branch. If I understand this correctly, the mesh itself will be fine, however, the GridFunction data will be permuted and with flipped signs in some cases. Is that correct?
If this is the case, this deserves to be mentioned in the CHANGELOG in a separate bullet.
This will also cause issues with the MFEM plugin in VisIt which is based on an older MFEM version.
The best solution will be to support reading correctly the previous default AMR mesh format and respective GridFunctions. How hard will this be to implement?

You are right, this could be a problem. The issue has already been addressed for the embedded Nodes, which are permuted when loading v1.1. We have unfortunately less control over other GridFunctions, but maybe if we remembered that the NCMesh was loaded from the legacy format, we could automatically convert other GridFunctions too. I'll check if this is possible.

You are right, this could be a problem. The issue has already been addressed for the embedded Nodes, which are permuted when loading v1.1. We have unfortunately less control over other GridFunctions, but maybe if we remembered that the NCMesh was loaded from the legacy format, we could automatically convert other GridFunctions too. I'll check if this is possible.

Done: all GridFunctions are now converted transparently if they are defined over an NCMesh that was loaded from a v1.1 file. The same happens for Nodes. Edges are now handled too, including a potential sign flip. Tested on GLVis compiled with this branch and different examples from master. Tried ex1 (with forced NC mesh) with cubic elements both in the solution and in Nodes. This indeed didn't work well, but works nice now. Tested both 2D and 3D and also ex3, all seems resolved.

Done: all GridFunctions are now converted transparently if they are defined over an NCMesh that was loaded from a v1.1 file. The same happens for Nodes. Edges are now handled too, including a potential sign flip. Tested on GLVis compiled with this branch and different examples from master. Tried ex1 (with forced NC mesh) with cubic elements both in the solution and in Nodes. This indeed didn't work well, but works nice now. Tested both 2D and 3D and also ex3, all seems resolved.

Re-merged in next for testing ...

Re-merged in next for testing ...

Re-merged in next.

Re-merged in next.

I just noticed that the reordering in this PR breaks previously saved files like the ones in
https://github.com/GLVis/web/tree/gh-pages/data
For example minimal-surface.saved and shaper.saved do not work correctly with glvis -saved or the web version in https://glvis.org/live.
Is there an easy way to update these files to the new ordering?

I just noticed that the reordering in this PR breaks previously saved files like the ones in
https://github.com/GLVis/web/tree/gh-pages/data
For example minimal-surface.saved and shaper.saved do not work correctly with glvis -saved or the web version in https://glvis.org/live.
Is there an easy way to update these files to the new ordering?

@tzanio, I thought the issue you are seeing was resolved in bed65fd3 but maybe not. @jakubcerveny can you take another look?

@tzanio, I thought the issue you are seeing was resolved in bed65fd3 but maybe not. @jakubcerveny can you take another look?

I rebuild everything again with glvis:master using 3 mfem hashes:

master
8509a8f (current par-ncmesh-saving-dev)
65c9a62  (par-ncmesh-saving-dev before the fix)

Here is what I get for ./glvis -saved minimal-surface.saved:
master

8509a8f

65c9a62

I rebuild everything again with glvis:master using 3 mfem hashes:

master
8509a8f (current par-ncmesh-saving-dev)
65c9a62  (par-ncmesh-saving-dev before the fix)

Here is what I get for ./glvis -saved minimal-surface.saved:
master

8509a8f

65c9a62

The vertex reordering seems OK (or there is none in the case of shaper.saved), but I'm suspecting there is another difference in the root_state which in v1.1 is not saved and has to be guessed (this is one of the motivations for the new format). I'll look more.

The vertex reordering seems OK (or there is none in the case of shaper.saved), but I'm suspecting there is another difference in the root_state which in v1.1 is not saved and has to be guessed (this is one of the motivations for the new format). I'll look more.

I'm suspecting there is another difference in the root_state which in v1.1 is not saved and has to be guessed (this is one of the motivations for the new format). I'll look more.

If it is really complicated, we can try to recreate these specific datafiles, but I'm concerned that users may have saved files like these e.g. from previous runs.

I'm suspecting there is another difference in the root_state which in v1.1 is not saved and has to be guessed (this is one of the motivations for the new format). I'll look more.

If it is really complicated, we can try to recreate these specific datafiles, but I'm concerned that users may have saved files like these e.g. from previous runs.

I'll find it and fix it, the ordering of elements is clearly different (Ctrl+O in GLVis). It's strange that the root states, the roots themselves and the entire trees all turned out to be exactly the same. But I'm closing in, there must be some simple explanation.

I'll find it and fix it, the ordering of elements is clearly different (Ctrl+O in GLVis). It's strange that the root states, the roots themselves and the entire trees all turned out to be exactly the same. But I'm closing in, there must be some simple explanation.

It's hopefully fixed. The problem was related to yet another inconsistency of the v1.1 format: since in master the loading code for v1.0 and v1.1 is intertwined, the leaf elements are loaded directly to the Mesh, in the order they appear in the file, regardless of the orientation of the NCMesh root elements. The proper SFC ordering is produced only after the NCMesh is modified and the leaves are exported to the Mesh again. This is of course a buggy behavior which this branch removes (the Mesh is now always generated from the NCMesh, no matter what), but for the purposes of loading the legacy v1.1 I have reintroduced this "feature".

It's hopefully fixed. The problem was related to yet another inconsistency of the v1.1 format: since in master the loading code for v1.0 and v1.1 is intertwined, the leaf elements are loaded directly to the Mesh, in the order they appear in the file, regardless of the orientation of the NCMesh root elements. The proper SFC ordering is produced only after the NCMesh is modified and the leaves are exported to the Mesh again. This is of course a buggy behavior which this branch removes (the Mesh is now always generated from the NCMesh, no matter what), but for the purposes of loading the legacy v1.1 I have reintroduced this "feature".

Thank you Jakub! I can confirm that 41089b9 fixed the issues I was seeing. 👍

Thank you Jakub! I can confirm that 41089b9 fixed the issues I was seeing. 👍

Re-merged in next.

Re-merged in next.

Re-merged in next for testing.

Re-merged in next for testing.

Hopefully one last thing that might save us trouble in the future: added a new (optional) section sfc_version (right after rank) that specifies the type of SFC ordering in the mesh file. Currently this needs to be zero (or unspecified), but in the future we might want to improve the SFC curve e.g. for triangles or prisms. Without this section, changing the SFC in MFEM would break older files. With the SFC version stored in the file, we will be able to fall back to an older ordering in the NCMesh class.

Hopefully one last thing that might save us trouble in the future: added a new (optional) section sfc_version (right after rank) that specifies the type of SFC ordering in the mesh file. Currently this needs to be zero (or unspecified), but in the future we might want to improve the SFC curve e.g. for triangles or prisms. Without this section, changing the SFC in MFEM would break older files. With the SFC version stored in the file, we will be able to fall back to an older ordering in the NCMesh class.

Re-merged in next.


[faucet] Add longer delay for timeout

Motivation
Faucet doesn't wait long enough for waiting on sequence number update to be processed, which results in 500 timeout errors frequently. To mitigate, change timeout to 4 seconds.
For application.client.delaybeforesend, when the libra CLI starts up, it prints out a whole bunch of help text, and if users’ development machines are slow enough, pexpect will start writing the mint command to stdin before the libra CLI is finished writing the help text, and so the help text will become interleaved with the mint command.  Then the help command finishes, and ends with a newline, which causes the (not yet finished) mint command to be processed by the CLI, which causes a parse error and the mint command is never successfully processed. This leads to the mint server returning 500's, with an error like the following:
buffer (last 100 chars): b'5b3e1e4 50000\r\n\r0000 LBR use_base_units\x1b[?2004l\r\nUnknown command: "m"\r\n\x1b[?2004h\x1b[6n\r\x1b[0Klibra% \r\x1b[7C'
The 0.1 second delay is there to help the issue, but it may not be enough -- so increasing it to 1s might mitigate the error here caused by CLI startup
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
CLI mint works

@bors-libra r+

:exclamation: Invalid command

Bors help and documentation

Bors is a Github App used to manage merging of PRs in order to ensure that CI is always green. It does so by maintaining a Merge Queue. Once a PR reaches the head of the Merge Queue it is rebased on top of the latest version of the PR's base-branch (generally master) and then triggers CI. If CI comes back green the PR is then merged into the base-branch. Regardless of the outcome, the next PR is the queue is then processed.
General

This project's Merge Queue can be found here.
This project requires PRs to be Reviewed and Approved before they can be queued for merging.
Before PR's can be merged they must be configured with "Allow edits from maintainers" enabled. This is needed for Bors to be able to update PR's in-place so that Github can properly recognize and mark them as "merged" once they've been merged into the upstream branch.

Commands
Bors actions can be triggered by posting a comment which includes a line of the form /<action>.

Command
Action
Description

Land
land, merge
attempt to land or merge a PR

Canary
canary, try
canary a PR by performing all checks without merging

Cancel
cancel, stop
stop an in-progress land

Cherry Pick
cherry-pick <target>
cherry-pick a PR into <target> branch

Priority
priority
set the priority level for a PR (high, normal, low)

Help
help, h
show this help message

Options
Options for Pull Requests are configured through the application of labels.

          Option
Description

Indicates that the PR is high-priority. When queued the PR will be placed at the head of the merge queue.

Indicates that the PR is low-priority. When queued the PR will be placed at the back of the merge queue.

Before merging the PR will be squashed down to a single commit, only retaining the commit message of the first commit in the PR.

@bors-libra land


SCKAN/Apinatomy version in map provenance

Needs updated map-knowledge with provenance() and model_provenance() methods for SciCrunch class.


Add config for gen-versions

Include config files that can be used with gen-verisons
so only one of the files needs to be specified instead of both.

This is targeted against release-v1.2, master already can build with updates to only EE or OS. After this is merged I expect to pick this to release-v1.1 and release-v1.0.

I also updated the generation template so it generates closer to what exists in the current version file.


Add dict obs support for PPO

Implementation of @kevinzakka's specification for dictionary observations. Dictionary-valued observations are useful in a lot of contexts: privileged critic inputs and policy inputs that mix pixels and state information to name two.
Behaviour

In the case of ndarray-valued observations, this PR makes no changes.
When you call ppo/train.py with a dict-valued observation with observation normalisation enabled, it is applied to obs['state'].
When you call any other training agent with a dict-valued observation, it raises a NotImplementedError. Supporting other agents would be future work.

Usage
An upcoming PR on pixels-based PPO training provides an example of how to train with dictionary observations. Essentially,

Ensure that env.step and env.reset return the observation in a dictionary form; see the example below.
Implement your environment's observation_size -> Union[int, Mapping[str, Tuple[int, ...]]] property.
Change the called network, the apply method and the dummy observation generation in make_policy_network and make_value_network
Implement your desired network to process the dictionary observation.

Example observation
obs = {
  'state': jnp.concat(data.qpos, data.qvel)
  'pixels/rgb': rgb/255.0
  'pixels/depth': depth/self._max_depth
}

Side-effects
AutoResetWrapper and ppo/losses.py needed to be slightly modified to support obs-dict training. These modifications do not appear to affect performance nor training (not shown), tested on the Franka Pick-up Cube task.

Obs Type
Commit f43727
PR

State
212118  SPS
213242 SPS

Dict
NA
212729 SPS

@Andrew-Luo1 how do AutoResetWrapper and losses need to be modified? We should probably apply those updates here as well?


Allow looping with all actions

I don't know if you're still maintaining this, but I have found this addition useful. It does not change any existing defined behaviour, and it adds (what I think is) useful behaviour.
This change allows the user to define an event from and to the same state, while also ensuring all the callbacks get called. It preserves the existing behaviour of an event which has a from state but no to state, which returns a NO_TRANSITION.

Q1: when this is useful? you have an example?
Q2: this is the only place where the magic state number used as a value, but setting it to 5 is it problematic when there is a state with number 5?
I think it would better test for nil (all the way through the fsm)?

Thanks for taking a look at this and asking the questions.
Is this useful? Do I have an example?
I am using it in this code here. This is part of an application released here. It runs on hardware with six buttons. Three are on/off keys (k1, k2, k3) and three are rotating encoders (e1, e2, e3). The application allows the user to record music, stop, play and move the record/play head backwards and forwards. It is ideal for a finite state machine.
One scenario for this application is that the user may be recording, realise they've made a mistake, and want to restart the recording immediately. I want them to be able to do this by simply hitting "record" again, not by having to hit "stop" first. Then if they do go from the record state to the record state (which I want to allow) the application has to tidy up from their first recording (a call to the leave_record() function) and then initialise a new recording (a call to the enter_record() function). The existing fsm code does not trigger these two functions, but my amendment does.
Can we do better than using a magic state 5? Wouldn't it be better to just check for nil all the way through fsm?
Yes, I agree it would be better to check for nil all the way through, and I have spent a bit of time understanding how to do that. Unfortunately I do think that will work. (You may be able to tell me I am wrong; I do not have much experience with Lua.)
One specific problem here is states_for_event, which maps from an event and a from state to a to state. If we allow states_for_event[event][from] == nil then we cannot distinguish between (a) a specific event that has a from state but no to state, and (b) a specific event that is not permitted from the from state. This becomes a problem in self.can(), for example.
So instead of checking for nil throughout fsm I can think of two solutions.
The first solution is to use a magic to state name to indicate a no-op (scenario (a) above). I proposed the magic name 5, but I agree this is not ideal. However, we do have * as a magic state name, so there is a precedent. Also, state names are intended to be used in function names, and there is already a limitation on permissible characters there (alphanumerics and underscore). So perhaps we can use a value outside of that limitation. Perhaps a single space or a # or similar would be acceptable instead of 5.
The second solution is to separately track all the no-op states. This might logically clear but would add a lot of code - an entirely new table that always needs to be checked. This would make fsm less readable and less maintainable.
So I would propose replacing 5 with a single space (or similar) to indicate a no-op event.

Certainly:

I saw a workaround for your specific situation. This was resolved by adding an extra state called "restart recording," which then triggers the next event automatically.

s:record -> e:k2_long_press -> s:restart_record -> e:restart -> s:record

Yes! A symbol that isn't permitted as a function name would be a more suitable choice :) Space isn't ideal, but other options like #@% would work better :)


Click support on range selection?

Hi,
Just wondering if there is click functionality on date range selection?
i.e. clicking on first and last dates of the range selection rather than click an drag over dates
Thanks

I'd like this one too. Think I'll have a go at this over the weekend and will update here if I have any luck. Would be nice to have the same function as the date range picker by @dangrossman :
https://github.com/dangrossman/bootstrap-daterangepicker

Hey @flash506
I have actually got somewhere with this......in a very basic way.
The requirement I had was to display the selected range and place them in 2 input fields. On the click of a 3rd date it chooses the range again.
Trying to work out how to show the start date, range and end date at the moment.
Is this of interest to you still?

Hi @dxbrich
Yeah still of interest to me :)
Thanks

Hi @flash506
Right. Here is my basic offering.
http://codepen.io/dxbrich/pen/MbYwZp
It doesn't highlight the chosen dates (yet) and you can choose an end date earlier than the start date at the moment. Work in progress ;)

Hi @dxbrich you finished this snippet with highlight?


https://developer.apple.com/xcode/resources/

https://github.com/swiftlang/swift-testing/commit/a4c26e35cdc0ce2daaff411f47ea6295a5042ddc

Completed


[Wish] Want to know if legend can be designed to be the same as github

As the question
Current Design

GitHub Design

It was available in v3, will be back before releasing final v4, as plugin

Ok, thank you.
Looking forward to the final release of v4.


OpenMP is missing!

ubuntu 14.04.5 LTS
ROS indigo
gcc version 5.4.1 20160904 (Ubuntu 5.4.1-2ubuntu1~14.04)
i already install sdk 2.0(https://github.com/IntelRealSense/librealsense/blob/development/doc/distribution_linux.md)
i want install  realsense-2.0.2 but i have some problem
**CMake Error at realsense-2.0.2/realsense_ros_camera/CMakeLists.txt:21 (message):
OpenMP is missing!
-- Configuring incomplete, errors occurred!**

Hello @kimkildae, please try to compile without OpenMP using the following line catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_OPENMP=false

This should be resolved (we no longer depend on OpenMP by default in librealsense)


Adds utility to reuse a single PowerShell session

Description
This PR adds the powershell utility that reuses a single session to execute commands over the plugin's lifetime.
Tests
The test-e2e-windows target passes locally, and following is the tail'd output.
2017/08/18 00:02:38 Creating volume [option_volume_247591] with options [ -o access=read-write] on VM [<IP_ADDRESS>]
2017/08/18 00:04:31 Checking volume [clone_src_volume_582587 option_volume_917152 option_volume_363414 option_volume_867138 option_volume_259466 option_volume_886045 option_volume_461698 option_volume_571036 option_volume_842558 option_volume_327807 option_volume_247591] availability from VM [<IP_ADDRESS>]
2017/08/18 00:04:32 END: VolumeCreateTestSuite.TestValidOptions
2017/08/18 00:04:32 Destroying volume [clone_src_volume_582587 option_volume_917152 option_volume_363414 option_volume_867138 option_volume_259466 option_volume_886045 option_volume_461698 option_volume_571036 option_volume_842558 option_volume_327807 option_volume_247591]
OK: 5 passed
--- PASS: Test (318.41s)
PASS
ok  	github.com/vmware/vsphere-storage-for-docker/tests/e2e	318.415s
make: Leaving directory `/go/src/github.com/vmware/vsphere-storage-for-docker/client_plugin'

what's the impact on time consumption ?

@msterin: @pshahzeb will be posting the stats.


Bug report: user profile not appearing in profile relationship dropdown.

To get expand working on a user profile, it must be assigned as a relation, but I cant seem to assign users through the admin panel

Screenshots attached above

Hi,
I've just tested it locally and I cannot reproduce the issue - the profiles records are visible and I'm able to select them.
Are you able to reproduce the issue on the live demo - https://pocketbase.io/demo/ (v0.3.2)?

I cannot reproduce the issue on the live demo, and the request that involve /api/collections are

when clicking on the item and opening the dropdown, this request is sent

but the dropdown is still empty

This couldn't possibly break it right?
Original code for user api

Modified user api

Changing the middleware for the user update action shouldn't be the cause for the missing dropdown items, but modifying the source is not the correct way to extend PocketBase. You should use only the hooks or create a custom action, otherwise you'll have to keep your project in sync manually with new updates.
On what PocketBase your changes are based? In v0.2.8 a fix was added to fix a similar issue with the user/relation fields - #100 (comment).
Have you made any other changes to the original API?
Are there any error logs in the browsers Inspect tool console?

No errors in the console, the only other change is the the embeded spa, and the version is based on the lastest pull from the repo

Could you try downloading the latest binary for your platform - https://github.com/pocketbase/pocketbase/releases/tag/v0.3.2
and start the executable inside it by pointing to your current pb_data directory, eg:
./pocketbase --dir="/path/to/your/pb_data"

If the admin that comes with the above executable shows the profiles correctly, then there is something in your changes that breaks the dropdown for some reason but I don't know how to debug or reproduce it (please make sure to check again that there are no js error/warning in the browser's console).

Could you try downloading the latest binary for your platform - https://github.com/pocketbase/pocketbase/releases/tag/v0.3.2 and start the executable inside it by pointing to your current pb_data directory, eg:
./pocketbase --dir="/path/to/your/pb_data"

If the admin that comes with the above executable shows the profiles correctly, then there is something in your changes that breaks the dropdown for some reason but I don't know how to debug or reproduce it (please make sure to check again that there are no js error/warning in the browser's console).

I will try that now, but the console is only just a few react dev tool warnings and slow internet warnings

I'm really confused.
Have you done any changes in the Admin UI? I cannot find the index.78281687.js from the above logs in the latest ui/dist (or in the last couple previous releases), which implies that you've manually built the ui, right?
What is loading the inject.js.map that results in 404; is there a inject.js script somewhere or is this from some browser extension?

Inject.js is from some extension that broke and the dev has not updated it yet, but disabling it gives the same result. Admin UI was rebuilt because of github actions (thats how I get my binary)

Just ran a clean download from github, and ran the pocketbase, still results in

Sorry, I don't know how to reproduce it.
A couple of things to try:

Open the admin in Incognito/Private mode in order to rule out that there is an extension that interferes with the UI.
Make sure profile relation fields is actually pointing to the user profiles collection
Try again with the latest release binary but this time with a clean pb_data and try to reproduce the issue

Tested in Chrome(android, incognito mode)

Testing if pb_data changes it, will update this comment accordingly

You could try running a SELECT * FROM _collections query via the sqlite3 cli against the "broken" db and post it here, eg.
sqlite3 pb_data/data.db "select * from _collections;"

You could try running a SELECT * FROM _collections query via the sqlite3 cli against the "broken" db and post it here, eg.
sqlite3 pb_data/data.db "select * from _collections"

60JgfaEFt44JljE|1|profiles|[{"system":true,"id":"ufalimoz","name":"userId","type":"user","required":true,"unique":true,"options":{"maxSelect":1,"cascadeDelete":true}},{"system":false,"id":"znfrpszc","name":"username","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"fmdt9v78","name":"avatar","type":"file","required":false,"unique":false,"options":{"maxSelect":1,"maxSize":5242880,"mimeTypes":["image/jpg","image/jpeg","image/png","image/svg+xml","image/gif"],"thumbs":null}},{"system":false,"id":"6u0qlcbr","name":"about","type":"text","required":false,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"d1nbswv6","name":"banner","type":"text","required":false,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"ezt3a4id","name":"firstname","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"ctc1d5sz","name":"lastname","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"k2d92izy","name":"group","type":"select","required":true,"unique":false,"options":{"maxSelect":1,"values":["admin","seller","user"]}},{"system":false,"id":"r2vdz2um","name":"friends","type":"user","required":false,"unique":false,"options":{"maxSelect":100,"cascadeDelete":false}},{"system":false,"id":"sounff4z","name":"sales","type":"number","required":false,"unique":false,"options":{"min":0,"max":null}},{"system":false,"id":"f7lfw0om","name":"discord","type":"text","required":true,"unique":true,"options":{"min":null,"max":null,"pattern":"^.{3,32}#[0-9]{4}$"}},{"system":false,"id":"qkutf7cb","name":"publicEmail","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"tmx1wbsg","name":"seller_status","type":"select","required":false,"unique":false,"options":{"maxSelect":1,"values":["normal","verified"]}}]||||userId = @request.user.id||2022-07-25 10:09:45.588|2022-07-27 10:30:32.852
i4rFwvg2DyA2muh|0|resources|[{"system":false,"id":"toyspaqr","name":"name","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"b6ill2rt","name":"price","type":"number","required":true,"unique":false,"options":{"min":null,"max":null}},{"system":false,"id":"vqg4l8qx","name":"icon","type":"file","required":true,"unique":false,"options":{"maxSelect":1,"maxSize":5242880,"mimeTypes":[],"thumbs":[]}},{"system":false,"id":"rfyvf978","name":"banner","type":"file","required":false,"unique":false,"options":{"maxSelect":1,"maxSize":5242880,"mimeTypes":[],"thumbs":[]}},{"system":false,"id":"wdrrzguf","name":"description","type":"text","required":false,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"mvuqs2bq","name":"content","type":"text","required":false,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"amnprmqc","name":"file","type":"relation","required":false,"unique":false,"options":{"maxSelect":20,"collectionId":"9tIzsBXLcEF66rM","cascadeDelete":false}},{"system":false,"id":"hehzcon9","name":"category","type":"select","required":true,"unique":false,"options":{"maxSelect":1,"values":["addons","eggs","scripts","services","plugins","themes","other"]}},{"system":false,"id":"g4xnerzz","name":"author","type":"user","required":true,"unique":false,"options":{"maxSelect":1,"cascadeDelete":false}},{"system":false,"id":"chokiklw","name":"profile","type":"relation","required":true,"unique":false,"options":{"maxSelect":1,"collectionId":"60JgfaEFt44JljE","cascadeDelete":false}}]||||||2022-07-25 11:24:58.643|2022-07-29 10:20:22.837
9tIzsBXLcEF66rM|0|files|[{"system":false,"id":"gwgwlraq","name":"filename","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"sjfa7lb5","name":"size","type":"number","required":true,"unique":false,"options":{"min":null,"max":null}},{"system":false,"id":"fassaige","name":"type","type":"text","required":true,"unique":false,"options":{"min":null,"max":null,"pattern":""}},{"system":false,"id":"pjisgbqt","name":"data","type":"file","required":true,"unique":false,"options":{"maxSelect":1,"maxSize":5242880,"mimeTypes":[],"thumbs":[]}},{"system":false,"id":"hzyau4ci","name":"uploader","type":"user","required":true,"unique":false,"options":{"maxSelect":1,"cascadeDelete":false}}]||||||2022-07-25 11:27:02.835|2022-07-25 11:27:02.835

The output seems OK - the resources are correctly wired to the profiles collection.
There seems to be something else going on but I can't think of anything else that could break it. If you have at least one profile record, then it should be visible in the relation field dropdown (you could try with creating another relation field pointing to a different collection).
I've tried it with several different configurations (including the schemas from above), but I'm not able to reproduce it.
@theMackabu Unless you are able to share a minimal reproducible example, I'm not sure that I can help you, so I'll close the issue for now as "Cannot reproduce".

The output seems OK - the resources are correctly wired to the profiles collection.
There seems to be something else going on but I can't think of anything else that could break it. If you have at least one profile record, then it should be visible in the relation field dropdown (you could try with creating another relation field pointing to a different collection).
I've tried it with several different configurations (including the schemas from above), but I'm not able to reproduce it.
@theMackabu Unless you are able to share a minimal reproducible example, I'm not sure that I can help you, so I'll close the issue for now as "Cannot reproduce".

other relations work fine, just profiles arent. Is it possible to nuke/wipe the profiles collection or transfer the two other collections to a fresh db?

I found how to reproduce it
I have created a friends relation with 100 or more user relations, then it will not appear in the menu

attached zip with demo database reproduced below
pb_data.zip

@theMackabu Thanks! I've found where the issue is and why I couldn't reproduce it previously. The reason for the bug is the group named field, because in the code when populating the select we are expecting the select option items to be either a flatten list with items or an array with the following structure [{group: 'My group', items: [...]}] and since the group key collides and ended up in the second case where the profile response doesn't have an items array and it doesn't show anything.
The code responsible for this is here - https://github.com/pocketbase/pocketbase/blob/master/ui/src/components/base/Select.svelte#L13-L33, but I'll have to think a little more how to fix it (one way will be to remove the implicit group conversion but I'll have to test it further and will let you know when I push a fix in master).

@theMackabu Thanks! I've found where the issue is and why I couldn't reproduce it previously. The reason for the bug is the group named field, because in the code when populating the select we are expecting the select option items to be either a flatten list with items or an array with the following structure [{group: 'My group', items: [...]}] and since the group key collides and ended up in the second case where the profile response doesn't have an items array and it doesn't show anything.
The code responsible for this is here - https://github.com/pocketbase/pocketbase/blob/master/ui/src/components/base/Select.svelte#L13-L33, but I'll have to think a little more how to fix it (one way will be to remove the implicit group conversion but I'll have to test it further and will let you know when I push a fix in master).

So it wasn't thr friends field,, good to know, and that makes sense

@theMackabu I've removed the implicit select items grouping - we are not using it anywhere and it just add unnecessary complexity to the reactivity of the select component.
The fix currently is only in the master branch, so you can pull the changes from there if  you want to test them.
I'm planning a v0.4.0 release sometime next week with other minor bugfixes (including query routing rewrite because of #204) and small improvements related to the realtime subscriptions.
Thanks again for isolating the bug!

Alright, thanks so much

@theMackabu Just to let you know that the fixes is also part of v0.3.4 release.


Charts values miscalculated with null values

Hello,
when I was writing a custom witness I was surprised to the values showed in the chart. For example the witness always setted the monitored value to 93
$this->setValue(93);
and the chart showed various values such 10.02, 23.43 etc
I finally discovered this was due to my tests when I didn't set any monitored value. It looks like null values in eyewitness_io_history_monitors table were also included in the computed displayed value.
Maybe this can occur in production whith unexpected exceptions, a quick workaround could be to use whereNotNull() on ChartTransformer generateCustom() function
$history = CustomHistory::where('meta', $witness->getSafeName())->whereNotNull('value')->get()->groupBy(function($item) {
but maybe there's a better place to take care of it.

Can I ask - in what situation would you sometimes set a value, and not set a value in other cases?
i.e. why not always set the value?

Well that happened writing a custom witness. At first I didn't set the values then I did every time so I guess it is not a real use case. But what happens when an uncaught exception is catched by Eyewitness ? The documentation says it will handle it like a fail but what about the exception is raised and no value setted ?

Yep - good point.
I'll make a change for this.

@curunoir - ok - the latest patch today should handle null values correctly and not use them as part of the average charting. Let me know how it goes :)

It looks good, thanks !


Generating a friendly name for policies

The displayed name of policies is generated via this piece of code for the policy profile form. It's basically utilizing ui_lookup for the left side which is not available in JS. Therefore, if I want to make the policy profile form API-driven, we need a way to construct this friendly name in the backend and provide it as a virtual column through the API.
@miq-bot assign @gtanzillo

@skateman It should be no problem to add a virtual column for the friendly name of a policy. I'm not able to find the controller code that is currently generating it. it seems the link above is outdated. Can you please point me to where that code is? Thanks.

@skateman It should be no problem to add a virtual column for the friendly name of a policy. I'm not able to find the controller code that is currently generating it. it seems the link above is outdated. Can you please point me to where that code is? Thanks.

    MiqPolicy.all.each do |p|
      @edit[:choices][ui_lookup(:model => p.towhat) + " #{p.mode.capitalize}: " + p.description] = p.id # Build a hash for the policies to choose from
    end

    MiqPolicy.all.each do |p|
      @edit[:choices][ui_lookup(:model => p.towhat) + " #{p.mode.capitalize}: " + p.description] = p.id # Build a hash for the policies to choose from
    end

@miq-bot assign @NickLaMuro

@miq-bot assign @NickLaMuro

As discussed, I think:

We should move the ui_lookup method into core, probably as a class method directly on the Dictionary class.  There is nothing really UI specific about what it does.
I'm good with having a way to do "friendly" or "display" names exposed as a virtual column, I would just recommend picking a method name that is consistent with any existing method names we already have.  At the class level we have display_name on models.  Not sure if we should also have a display_name at the instance level that does what you are proposing, or have the method named something else.  Perhaps @mzazrivec has some recommendations here.
ui_lookup(:model => p.towhat) + " #{p.mode.capitalize}: " + p.description doesn't seem very i18n compliant.

Side question for @mzazrivec .  What is the difference between model.display_name and ui_lookup(:model => model), and when should we use one over the other?

As discussed, I think:

We should move the ui_lookup method into core, probably as a class method directly on the Dictionary class.  There is nothing really UI specific about what it does.
I'm good with having a way to do "friendly" or "display" names exposed as a virtual column, I would just recommend picking a method name that is consistent with any existing method names we already have.  At the class level we have display_name on models.  Not sure if we should also have a display_name at the instance level that does what you are proposing, or have the method named something else.  Perhaps @mzazrivec has some recommendations here.
ui_lookup(:model => p.towhat) + " #{p.mode.capitalize}: " + p.description doesn't seem very i18n compliant.

Side question for @mzazrivec .  What is the difference between model.display_name and ui_lookup(:model => model), and when should we use one over the other?

Side question for @mzazrivec . What is the difference between model.display_name and ui_lookup(:model => model), and when should we use one over the other?

Short answer: class.display_name() is a pluggable implementation of ui_lookup(:model => model). .display_name is called on a class, while ui_lookup() and Dictionary.gettext() look into locale/en.yml dictionary.
My plan was to modify the internals of Dictionary.gettext() so that it uses the pluggable .display_name()method and then we can ditch the :model subtree in locale/en.yml. Though to make the transition seamless, I have to get rid of some ugly hacks that we had in this area (work in progress ATM).
When to use which? It should be perfectly safe to use Vm.display_name instead of ui_lookup(:model => 'Vm') even now. In the future, i.e. once I make the switch, both will use the same thing.

Side question for @mzazrivec . What is the difference between model.display_name and ui_lookup(:model => model), and when should we use one over the other?

Short answer: class.display_name() is a pluggable implementation of ui_lookup(:model => model). .display_name is called on a class, while ui_lookup() and Dictionary.gettext() look into locale/en.yml dictionary.
My plan was to modify the internals of Dictionary.gettext() so that it uses the pluggable .display_name()method and then we can ditch the :model subtree in locale/en.yml. Though to make the transition seamless, I have to get rid of some ugly hacks that we had in this area (work in progress ATM).
When to use which? It should be perfectly safe to use Vm.display_name instead of ui_lookup(:model => 'Vm') even now. In the future, i.e. once I make the switch, both will use the same thing.

Closing this as I think the above PR that was merged in Feb solves this issue.


Update inaccurate numbering in the README

I noticed the numbering in the README is a bit misleading. I raised this PR to fix the inaccurate numbering.
Screenshots
Before:

After:

thanks


Boris

Hello.I am trying to compile sketch for arducam esp8266 uno board ,but I get the message "Error compiling for board ArduCAM ESP8266 UNO",even though I've installed apropriate board on board manager with this address "http://www.arducam.com/downloads/ESP8266_UNO/package_ArduCAM_index.json".
I cannot find what seems to be a problem?

@borisg9, would you please show me the error message?


Developer Documentation

Please add documentation to either the wiki or the readme.md to help us know how best to get setup and test our changes before we submit a pull request.
As of right now, the wiki redirects to the main page.

Any update, @ashishpd ?

submit PR #17 to include the azure resources structure, you need to create required resources and deploy the app services to let run the app.

@zyhfish Just to confirm the two other databases are just blank and used by the service?  Or do we need to have a DNN install on the one?

Hi @mitchelsellers , as I checked the source code, the other 2 databases should be used during the package verification process, the process will maintenance the DB objects.

Great!
Can you confirm the sizing/configuration of each of these elements as they are today for EVS?  I want to start getting resources provisioned for us.

evsdatabase: Standard S0 10 DTU
evstestinstall01: Basic
evstestrestore01: Basic
evswebapiservice: Exra Small
evswebsitev2: B1:1
evsworker01: Extra Small


Refactor BaseEnvironment

This will close #3643
Also related to #3665

I have removed the record dependency from BaseEnvironment and BuildCommand, but still there is core repetition, I will continue refactoring it later. For now, all tests pass :)

What I done so far:

Remove record dependency from BuildCommand and BaseEnvironment
Use pre_run_command and post_run_command to extend the behavior of the run method.
Use a mixin to add the record command behavior.
Previous run accepted record=True, record_as_success=False and warn_only=False. This variables had a dependence between each other. What I propose is to use only record=True and warn_only=False.

What is missing:

Change method calls to new vars (record=True and warn_only=False).
Maybe update/add tests.

I just update this PR to the current master, but I want to wait till #3842 is fixed to continue.

@stsewd issues mentioned "as blocking" this PR are already closed/merged.
Do you want to continue with this PR? I think it was close enough to be merged.

@humitos turns out I forgot everything about the current state of this PR :cry: but I have read all the related code again, and there is a part blocking this. We were relying on the record param to change the value of warn_only and friends, but now BaseEnvironment shouldn't have any idea about record, so we need to remove that magical dependency and not change any other parameters based on record. I was doing that here but looks like it's big, so I'm going to do another PR.

Ok, I think I got it, after reading https://github.com/rtfd/readthedocs.org/pull/3520/, but I'm not sure. We are lying to the users when saving the result to something else. But I understand that would make the build to appear like a failure... But, if the command failed, then something isn't right. Also, we are just using that to execute some commands in the VCS, actually we don't use record_as_success on git anymore.

I was checking, we still need that for svn and bzr :/, I guess I'll continue here


Provide option to delete previous manifests

Description of the new feature/enhancement
Provide the option to or automatically remove previous manifests if either

The URL's for the new version are the same as the previous version
The previous version URL's return 404's

Proposed technical implementation details (optional)

Before we begin the work on this; We need to understand how wingetbot handles this, as well as how to properly create the PR's so that they are correctly processed in the pipeline

The previous version URL's return 404's

URLs are automatically scanned by wingetbot at regular intervals so I don't think we need this.

Is it already done?

No, not completely


ImportError: No module named '_pvcnn_backend'

ros@rosmachine:~/Downloads/pvcnn$ python3 train.py configs/kitti/frustum/pvcnne.py --devices 0 --evaluate --configs.evaluate.best_checkpoint_path /home/ros/Downloads/kitti.frustum.pvcnne.pth.tar --configs.evaluate.num_tests 20
==> loading configs from ['configs/kitti/frustum/pvcnne.py']
Traceback (most recent call last):
File "train.py", line 266, in
main()
File "train.py", line 79, in main
configs = prepare()
File "train.py", line 24, in prepare
configs.update_from_modules(*args.configs)
File "/home/ros/Downloads/pvcnn/utils/config.py", line 113, in update_from_modules
importlib.import_module(module)
File "/usr/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 953, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/ros/Downloads/pvcnn/configs/kitti/frustum/init.py", line 7, in
from meters.kitti import MeterFrustumKitti
File "/home/ros/Downloads/pvcnn/meters/kitti/init.py", line 1, in
from meters.kitti.frustum import MeterFrustumKitti
File "/home/ros/Downloads/pvcnn/meters/kitti/frustum.py", line 4, in
from modules.frustum import get_box_corners_3d
File "/home/ros/Downloads/pvcnn/modules/init.py", line 1, in
from modules.ball_query import BallQuery
File "/home/ros/Downloads/pvcnn/modules/ball_query.py", line 4, in
import modules.functional as F
File "/home/ros/Downloads/pvcnn/modules/functional/init.py", line 1, in
from modules.functional.ball_query import ball_query
File "/home/ros/Downloads/pvcnn/modules/functional/ball_query.py", line 3, in
from modules.functional.backend import _backend
File "/home/ros/Downloads/pvcnn/modules/functional/backend.py", line 8, in
sources=[os.path.join(_src_path,'src', f) for f in [
File "/home/ros/.local/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 898, in load
is_python_module)
File "/home/ros/.local/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1097, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
File "/home/ros/.local/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1418, in _import_module_from_library
file, path, description = imp.find_module(module_name, [path])
File "/usr/lib/python3.7/imp.py", line 296, in find_module
raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named '_pvcnn_backend'
Facing this issue while trying to do inference ,can anyone help me to solve this issue.

I'm not sure whether the issue is related to ROS. Could you try the installation on a normal machine?

I'm closing this issue due to inactivity. Please feel free to reopen it if the problem has not been resolved.

I also encountered the same problem. How should I solve it?
(pvcnn) root@stoneyshi-LC0:~/mycode/pvcnn$ python train.py configs/s3dis/pvcnn/area5.py --devices 0,1 --evaluate --configs.evaluate.best_checkpoint_path s3dis.pvcnn.area5.c1.pth.tar ==> loading configs from ['configs/s3dis/pvcnn/area5.py'] Traceback (most recent call last):   File "train.py", line 266, in <module>     main()   File "train.py", line 79, in main     configs = prepare()   File "train.py", line 24, in prepare     configs.update_from_modules(*args.configs)   File "/home/shiying/mycode/pvcnn/utils/config.py", line 113, in update_from_modules     importlib.import_module(module)   File "/home/shiying/anaconda3/envs/pvcnn/lib/python3.7/importlib/__init__.py", line 127, in import_module     return _bootstrap._gcd_import(name[level:], package, level)   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import   File "<frozen importlib._bootstrap>", line 983, in _find_and_load   File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import   File "<frozen importlib._bootstrap>", line 983, in _find_and_load   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked   File "<frozen importlib._bootstrap_external>", line 728, in exec_module   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed   File "/home/shiying/mycode/pvcnn/configs/s3dis/pvcnn/__init__.py", line 3, in <module>     from models.s3dis import PVCNN   File "/home/shiying/mycode/pvcnn/models/s3dis/__init__.py", line 1, in <module>     from models.s3dis.pointnet import PointNet   File "/home/shiying/mycode/pvcnn/models/s3dis/pointnet.py", line 4, in <module>     from models.utils import create_pointnet_components, create_mlp_components   File "/home/shiying/mycode/pvcnn/models/utils.py", line 5, in <module>     from modules import SharedMLP, PVConv, PointNetSAModule, PointNetAModule, PointNetFPModule   File "/home/shiying/mycode/pvcnn/modules/__init__.py", line 1, in <module>     from modules.ball_query import BallQuery   File "/home/shiying/mycode/pvcnn/modules/ball_query.py", line 4, in <module>     import modules.functional as F   File "/home/shiying/mycode/pvcnn/modules/functional/__init__.py", line 1, in <module>     from modules.functional.ball_query import ball_query   File "/home/shiying/mycode/pvcnn/modules/functional/ball_query.py", line 3, in <module>     from modules.functional.backend import _backend   File "/home/shiying/mycode/pvcnn/modules/functional/backend.py", line 8, in <module>     sources=[os.path.join(_src_path,'src', f) for f in [   File "/home/shiying/anaconda3/envs/pvcnn/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 997, in load     keep_intermediates=keep_intermediates)   File "/home/shiying/anaconda3/envs/pvcnn/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1213, in _jit_compile     return _import_module_from_library(name, build_directory, is_python_module)   File "/home/shiying/anaconda3/envs/pvcnn/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1560, in _import_module_from_library     file, path, description = imp.find_module(module_name, [path])   File "/home/shiying/anaconda3/envs/pvcnn/lib/python3.7/imp.py", line 296, in find_module     raise ImportError(_ERR_MSG.format(name), name=name) ImportError: No module named '_pvcnn_backend'

Could you please try to add verbose=True to https://github.com/mit-han-lab/pvcnn/blob/master/modules/functional/backend.py#L6, which should give you more detailed logging information?


unexpected failure of obc_tests

https://github.com/envoyproxy/envoy-mobile/runs/5744148680?check_suite_focus=true
INFO: Repository bazel_gazelle_go_repository_tools instantiated at:
38
/Users/runner/work/envoy-mobile/envoy-mobile/WORKSPACE:40:25: in
39
/private/var/tmp/_bazel_runner/b225883c33850f3f191a570e7b70ec29/external/envoy/bazel/dependency_imports.bzl:21:25: in envoy_dependency_imports
40
/private/var/tmp/_bazel_runner/b225883c33850f3f191a570e7b70ec29/external/bazel_gazelle/deps.bzl:75:24: in gazelle_dependencies
41
Repository rule go_repository_tools defined at:
42
/private/var/tmp/_bazel_runner/b225883c33850f3f191a570e7b70ec29/external/bazel_gazelle/internal/go_repository_tools.bzl:117:38: in
43
ERROR: An error occurred during the fetch of repository 'bazel_gazelle_go_repository_tools':
44
Traceback (most recent call last):
45
File "/private/var/tmp/_bazel_runner/b225883c33850f3f191a570e7b70ec29/external/bazel_gazelle/internal/go_repository_tools.bzl", line 103, column 13, in _go_repository_tools_impl
46
fail("failed to build tools: " + result.stderr)
47
Error in fail: failed to build tools: Timed out
48
ERROR: /Users/runner/work/envoy-mobile/envoy-mobile/WORKSPACE:40:25: fetching go_repository_tools rule //external:bazel_gazelle_go_repository_tools: Traceback (most recent call last):
49
File "/private/var/tmp/_bazel_runner/b225883c33850f3f191a570e7b70ec29/external/bazel_gazelle/internal/go_repository_tools.bzl", line 103, column 13, in _go_repository_tools_impl
50
fail("failed to build tools: " + result.stderr)
51
Error in fail: failed to build tools: Timed out
52
ERROR: /private/var/tmp/_bazel_runner/b225883c33850f3f191a570e7b70ec29/external/com_envoyproxy_protoc_gen_validate/BUILD:18:11: no such package '@com_github_lyft_protoc_gen_star//': no such package '@bazel_gazelle_go_repository_config//': no such package '@bazel_gazelle_go_repository_tools//': failed to build tools: Timed out and referenced by '@com_envoyproxy_protoc_gen_validate//:protoc-gen-validate_lib'
53
ERROR: Analysis of target '//test/objective-c:envoy_bridge_utility_test' failed; build aborted:
54
INFO: Elapsed time: 1055.588s
55
INFO: 0 processes.
56
FAILED: Build did NOT complete successfully (161 packages loaded, 10678 targets configured)
57
ERROR: Couldn't start the build.

It's a terrible error message. My theory is that it's pulling a part of the bazel-gazelle integration from a random place on the internet that was temporarily unavailable (it would be great for Bazel to tell us which place). I think caching the files in a more reliable place might help - coincidentally, I recently worked on something that we may be able to use here, once it's production ready.

Yeah, a lot of the EM CI failures I've seen have either been flaky tests (which we'll work more on) or fetch fails.  If we can cache more files that'd be fantastic!


NER模型加载问题

Describe the bug
A clear and concise description of what the bug is.
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
import hanlp

HanLP = hanlp.load(hanlp.pretrained.ner.MSRA_NER_BERT_BASE_ZH)

result = HanLP(['2021年HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。', '阿婆主来到北京立方庭参观自然语义科技公司。'])

print(result)

Describe the current behavior
2024-02-27 10:35:41.821627: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Failed to load https://file.hankcs.com/hanlp/ner/ner_bert_base_msra_20211227_114712.zip
If the problem still persists, please submit an issue to https://github.com/hankcs/HanLP/issues
When reporting an issue, make sure to paste the FULL ERROR LOG below.
================================ERROR LOG BEGINS================================

Expected behavior
A clear and concise description of what you expected to happen.
System information

OS: Windows-10-10.0.22621-SP0
Python: 3.9.0
PyTorch: 2.2.1+cpu
TensorFlow: 2.15.0
HanLP: 2.1.0-beta.56

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

[x] I've completed this form and searched the web for solutions.

When reporting an issue, make sure to paste the FULL ERROR LOG below.


Crash on closing a network file

Prerequisites

[ ] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: http://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: http://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
Atom app crashed when i saved the file in a network and then close the file. Tried this on Six(6) different network paths and all of them have the same effect.
Steps to Reproduce

Open atom app and create a new file.
Type something in the file and save it in a network path
Close the file by clicking the middle button of the mouse on top of the file name
App crashed

Expected behavior: Close the file without crashing
Actual behavior: Atom app crashed
Reproduces how often: 100%
Versions
You can get this information from copy and pasting the output of atom --version and apm --version from the command line. Also, please include the OS and what version of the OS you're running.
Atom    : 1.17.2
Electron: 1.3.15
Chrome  : 52.0.2743.82
Node    : 6.5.0
Windows 10
Additional Information
Any additional information, configuration or data that might be necessary to reproduce the issue.

Thanks for the report!  Can you confirm a few things for us?

Did this behavior just start recently (e.g. after updating to 1.17.2)?
It looks like the checkbox in the Prerequisites section of the issue template isn't checked - can you reproduce the error in safe mode (atom --safe)?
Can you share any crash logs that you find (http://flight-manual.atom.io/hacking-atom/sections/debugging/#find-crash-logs)?


Support for input-group containing form-floating elements in Boostrap 5

I would love to use form-floating elements in an input group but ran into issues.
Both in Firefox (87.0 (64-bit) Ubuntu) and Chrome (Version 90.0.4430.72 (Official Build) (64-bit) Ubuntu), the floating label disappears when the input field is active. Also, on Chrome, the initial empty state is already showing the floating label above the input field.
Here is a minimal example using the current Bootstrap 5.0 version:
https://codepen.io/oatsandcoffee/pen/oNBQPJW
My apologies if this is already known or expected behavior.
Thanks!

you must include containers

        <input type="password" class="form-control" id="input-password" placeholder="Password">
        Password

    <button type="button" class="btn btn-primary">Save</button>

Thank you. I could swear it didn't work when I tried it first. Anyhow, thank you!

@WinterSilence , The border radius of the input field is not attached to the buttons/.input-group-text span properly.


Copied play/level files to ozaria folder

Copied the play level view's style files from /app/styles folder to /ozaria/site/styles folder.
Moved /ozaria/site/templates/play/ files to /ozaria/site/templates/play/level/, and updated their references

Me too 👍 The one area I noticed a conflict before was play/level/tome and that looks good!


The OfflineCache is Caching my RestFul API

Hi
I get problem when I use OfflineCache 😢
I have spring boot restful api server, the request work fine but the response I get the cache of my restful from your OfflineCache.
when I comment Offline Cache it work fine.
example of restful cache file name "http%3A%2F%2Flocalhost%2Flogin%2Fauthorize%2F"
it like "http://localhost/login/authorize/"   <== my api   😢
Thanks for this wondrful library.
✌😁🌹

In order to do the offline caching, mapjfx needs to intercept all the network calls, there is no way to only intercept the calls from the mapjfx component.
The OfflineCache class has a method setNoCacheFilters(List<String>). With this method you can set a list of Strings - which will be compiled to java.util.regex.Patterns - for which the caching is disabled. Any URL that matches one of these patterns will not be cached.
Currently you can only set patterns for URLS that should not be cached.

thanks for replay
I will use setNoCacheFilters(List)
but, can you make method that is opposite of setNoCacheFilters(List)
because you we can handle map url better than handle all other urls
also, I don't know if that intercept WebView or not
🌹

put that in issue #69

implemented in 2.10.2 and 1.29.2, OfflineCache now has a method setCacheFilters(List<String>) which defines what to cache.


METRON-857 FIX: Full_Dev regression: use the profiles to inject the package phase resource copy

Contributor Comments
First, I am very sorry to have broken Full_Dev.
I believe the issue is the order of execution within the package phase when using profiles to inject executions.
Simliar to :
http://stackoverflow.com/questions/22150209/maven-changes-the-order-of-plugins-of-different-profiles
https://issues.apache.org/jira/browse/MNG-2258
To address this, I have injected the copy execution as well as the execute plugin in the profile.  I believe doing this will maintain the correct order, and in my testing spinning up Full Dev a few times it has.
This is building in travis as we speak, but I'm posting the PR to get this tested by someone else and reviewed asap.
###Testing

spin up full dev
verify that it installs
vagrant ssh & ls /localrepo to verify the rpms are deployed

Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[ x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[ x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[ x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x ] Have you included steps or a guide to how the change may be verified and tested manually?

[in progress ] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via:
mvn -q clean integration-test install && build_utils/verify_licenses.sh

[NA ] Have you written or updated unit tests and or integration tests to verify your changes?

[ NA] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[ x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[ x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
bin/generate-md.sh
mvn site:site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

I'll run it through its paces today and let you know. Thanks!

Appreciate it, sorry for the trouble

Ran up full dev with this change and it is working again.  Thanks for addressing this so quickly @ottobackwards. +1

Ditto. +1, thanks Otto!

lol beat me to it


Module does not compile on node v.12+

We really would like to update our application stack to the latest version on node, but this module is holding us back. Trying on anything greater than v0.10 results in the following output:
npm ERR<EMAIL_ADDRESS>install: make build && make install
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'make build && make install'.
npm ERR! This is most likely a problem with the db-nuodb package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     make build && make install
npm ERR! You can get their info via:
npm ERR!     npm owner ls db-nuodb
npm ERR! There is likely additional logging output above.

+1

+1

+100000000000000000000000 we really need a driver for nodejs !

Yeah @PNE1 , we needed it so badly that we wrote the db querying portion in Python (there is a working Python module for Nuodb), and had our node app call that python script...  sure would have been nice to just use node!