reference custom tflite model from inside graph definition

Plugin Version or Commit ID
6b8c6743f23539f7604e74dc260b01e0f58f1707
Unity Version
2020.3.26f1
Your Host OS
Windows 10 Pro
Target Platform
UnityEditor
Description
I'd like to adapt the HelloWorld Scene with a graph that:

accepts a simple float array as input
converts the array into a tflite tensor
runs a tflite model
returns the output of the network (likewise a float array)

While this little toy example runs fine in mediapipe with some example driver code in c++, I cannot get it working in MediaPipeUnityPlugin.
In C++ I just had to specify a relative path to the tflite model in the TfLiteInferenceCalculator section of the graph definition
model_path: ""mediapipe/models/my_model.tflite""
Obviously this line has to be changed when switching to Unity, but how?
What are the exact steps to reference the tflite model correctly?
Code to Reproduce the issue
The graph definition looks like this:
var configText = @"
        input_stream: ""MATRIX:in""
        output_stream: ""FLOATS:out""

        node
        {
          calculator: ""TfLiteConverterCalculator""
          input_stream: ""MATRIX:in""
          output_stream: ""TENSORS:image_tensor""
          options:
          {
              [mediapipe.TfLiteConverterCalculatorOptions.ext]
              {
              zero_center: false
              }
          }
        }

        node
        {
          calculator: ""TfLiteInferenceCalculator""
          input_stream: ""TENSORS:image_tensor""
          output_stream: ""TENSORS:tensor_features""
          options:
          {
            [mediapipe.TfLiteInferenceCalculatorOptions.ext]
            {
              model_path: ""mediapipe/models/my_model.tflite""
            }
          }
        }

        node
        {
          calculator: ""TfLiteTensorsToFloatsCalculator""
          input_stream: ""TENSORS:tensor_features""
          output_stream: ""FLOATS:out""
        }
      ";

Here is the (not yet tested) driver code of the above graph:
      var graph = new CalculatorGraph(configText);
      var poller = graph.AddOutputStreamPoller<float[]>("out").Value();

      // load tflite model from assets
     // ... (not sure what to do here)

      Debug.Log("StartRun");
      graph.StartRun().AssertOk();
      for (var i = 0; i < 10; i++)
      {
        var floatArray = new float[6] { 0, 1, 2, 3, 4, 5 };
        var input = new FloatArrayPacket(floatArray, new Timestamp(i));
        graph.AddPacketToInputStream("in", input).AssertOk();
      }
      graph.CloseInputStream("in").AssertOk();

      Debug.Log("Poll output");
      var output = new FloatArrayPacket();
      while (poller.Next(output))
      {
        Debug.Log(output.Get());
      }

      graph.WaitUntilDone().AssertOk();
      graph.Dispose();

      Debug.Log("Done");
    }

Additional Context
I tried

to rename my_model.tflite to StreamingAssets folder (where apparantly all other tflite model reside) and rename it to my_model.bytes but it's still unclear to me how to reference it from there in the graph definition.

Thanks for the quick reply! This is useful information for a higher level understanding of the Plugin.
Meanwhile I found a hack to load the model at least when running the graph in UnityEditor.
It appears that my next problem is that the graph expects an Eigen::Matrix rather than a vector of floats.
MediaPipeException: INVALID_ARGUMENT: Graph has errors:
Packet type mismatch on calculator outputting to stream "in": The Packet stores "float [0]", but "class Eigen::Matrix<float,-1,-1,0,-1,-1>" was requested.
Mediapipe.Status.AssertOk () (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/Port/Status.cs:48)
Mediapipe.Unity.HelloWorld.Start () (at Assets/MediaPipeUnity/Samples/Scenes/Tutorial/HelloWorld.cs:72)

So far I did not figure out to Eigen::Matrix as input package, but I guess this is for a different issue...

To pass Eigen::Matrix to MediaPipe, we need a Packet<Eigen::Matrix> class (in C#), but there are no such classes currently.
cf. https://github.com/homuler/MediaPipeUnityPlugin/wiki/API-Overview#packett
So you need to implement it (PR is welcome!).
It may be convenient to implement a function that converts mediapipe::MatrixData (in practice, it should be serialized to const char*) into mediapipe::Packet<Eigen::Matrix> since it's easy to handle protocol buffer classes in C# and it seems that Eigen::Matrix can be derived from mediapipe::MatrixData easily with mediapipe::MatrixFromMatrixDataProto.
I think an example from Anchor3dVectorPacket would be helpful in this case (you may want to implement MpReturnCode mp__MakeMatrixPacket__PKc_i(const char* matrix_data, int size, mediapipe::Packet** packet_out)).
https://github.com/homuler/MediaPipeUnityPlugin/blob/d70c6f32c61ef4a199f04c580e75337038e746be/mediapipe_api/graphs/instant_motion_tracking/calculators/transformations.cc#L11-L32
https://github.com/homuler/MediaPipeUnityPlugin/blob/d70c6f32c61ef4a199f04c580e75337038e746be/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/Packet/Anchor3dVectorPacket.cs#L22-L33
You can deserialize const char* to the corresponding protocol buffer class.
https://github.com/homuler/MediaPipeUnityPlugin/blob/d70c6f32c61ef4a199f04c580e75337038e746be/mediapipe_api/framework/calculator_graph.cc#L27-L33
https://github.com/homuler/MediaPipeUnityPlugin/blob/d70c6f32c61ef4a199f04c580e75337038e746be/mediapipe_api/framework/calculator_graph.cc#L11-L16

then I would need one more CalculatorNode in the graph that converts my custom Protobuf matrix into an Eigen::Matrix !?

in that case it might be easier to write a calculator that converts the already existing FloatArrayPacket into an Eigen::Matrix

No.
The error you reported in https://github.com/homuler/MediaPipeUnityPlugin/issues/632#issuecomment-1171430852 occurred because you sent a FloatArrayPacket to the in stream. By implementing MatrixPacket : Packet<MatrixData> (in C#) that manages mediapipe::Packet which holds Eigen::Matrix internally (in C++), it can be resolved (note that mediapipe is a namespace used in C++ and Mediapipe is a namespace used in C#).

While I can create a MatrixData matrix = new MatrixData(); object, I don't know how to 'fill it' with data. The only constructor I can find expects an already existing MatrixData object and clones data from there.

You can use setters to set values.
See the official protocol buffer document for more details.
To pass MatrixData to the Unmanaged Code, serialize it like this.
https://github.com/homuler/MediaPipeUnityPlugin/blob/d70c6f32c61ef4a199f04c580e75337038e746be/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/CalculatorGraph.cs#L31

Can all this be done in C#?

No. MpReturnCode mp__MakeMatrixPacket__PKc_i(const char* matrix_data, int size, mediapipe::Packet** packet_out)) must be implemented in C++.
This function takes matrix_data, which is a serialized MatrixData (byte[] in C#).
In that function, I think you can convert matrix_data into Eigen::Matrix using mediapipe::MatrixFromMatrixDataProto, then build mediapipe::Packet that holds Eigen::Matrix insternally.
The pointer to the built mediapipe::Packet needs to be set to packet_out.

So I will

add a set of C++ functions which will be exposed to C# similar to

medipipe_api/.../transformations.cc, transformations.h
for example a pair:

medipipe_api/.../matrix_frame.cc, matrix_frame.h
the minimum required functions in those files are similar to those in transformations.cc (?), i.e. :

MpReturnCode mp__MakeAnchor3dVectorPacket__PA_i(const mediapipe::Anchor3d* value, int size, mediapipe::Packet** packet_out)`
MpReturnCode mp__MakeAnchor3dVectorPacket_At__PA_i_Rt(const mediapipe::Anchor3d* value, int size, mediapipe::Timestamp* timestamp, mediapipe::Packet** packet_out)

not sure what __PA_i an _At__PA_i_Rt stands for (?), but it seems the first function creates a packet without timestamp and the second one with timestamp (which is propably what I need)

I will create a function to send data from C# to C++

for example (as you mentioned) a function

 MpReturnCode mp__MakeMatrixPacket__PKc_i(const char* matrix_data, int size, mediapipe::Packet** packet_out))

-> wouldn't it be better / easier to send some kind of float array rather than const char* ? should be faster and easier for conversion or am I getting something wrong?

not sure what __PA_i an _At__PA_i_Rt stands for (?),

When porting, I make it a habit to add a suffix that is similar to the C++ mangling rules to distinguish functions with the same name (e.g. mediapipe::CalculatorGraph() -> mp_CalculatorGraph__, mediapipe::CalculatorGraph(const CalculatorGraphConfig& config) -> mp_CalculatorGraph__PKc_i).
I've not defined the rule strictly, but in this case,

MakeAnchor3dVectorPacket represents mediapipe::MakePacket<Anchor3d>
MakeAnchor3dVectorPacket_At represents mediapipe::MakePacket<Anchor3d>.At(mediapipe::Timestamp)
PA represents a pointer to Anchor3d (the type of value)
i reprensets int (the type of size, i)
Rt represents a reference to Timestamp (the type of timestamp)

wouldn't it be better / easier to send some kind of float array rather than const char* ? should be faster and easier for conversion or am I getting something wrong?

Maybe.
To initialize a MatrixPacket, you need to pass a value that represents Eigen::Matrix, and I thought Mediapipe.MatrixData would be convenient.
If you prefer float[] or float[][], it would be OK.
Which one is easier would depend on the person. Assuming mediapipe::MatrixFromMatrixDataProto, I thought Eigen::Matrix could be handled more easily than float[].
Probably float[] version would be faster, but const char* is actually a protocol buffer's serialized data, so I don't think float[] is clearly faster than the other.
P.S. You don't need to implement mp__MakeMatrixPacket__PKc_i in C#.
https://github.com/homuler/MediaPipeUnityPlugin/blob/d70c6f32c61ef4a199f04c580e75337038e746be/Packages/com.github.homuler.mediapipe/Runtime/Scripts/PInvoke/NativeMethods/Graphs/InstantMotionTracking_Unsafe.cs#L14-L18

regarding the second part of my question "how to fill the MatrixData object"  I guess I found the answer. It seem like the equivalent of this vector
var floatArray = new float[6] { 0, 1, 2, 3, 4, 5 };

is a MatrixData object filled via .PacketData.Add like this:
var matrix = new MatrixData();
matrix.PackedData.Add(0);
matrix.PackedData.Add(1);
matrix.PackedData.Add(2);
matrix.PackedData.Add(3);
matrix.PackedData.Add(4);
matrix.PackedData.Add(5);

Just in case you are interested, this is work in progress

@mgarbade the link is giving me a 404 page I can’t access it I think the repo is private

Just in case you are interested, this is work in progress

ups, sorry. I just repaired the link. it should work now

It seems like I cannot convert a const char * into a MatrixData object. I assume that I'm doing the exact same as [this snippet] (https://github.com/google/mediapipe/blob/e6c19885c6d3c6f410c730952aeed2852790d306/mediapipe/examples/desktop/youtube8m/extract_yt8m_features.cc#L66-L77) from the official mediapipe repo.
However in my case it only compiles if I use mediapipe::file::GetContents to write the serialized matrix data into a string:
MpReturnCode mp__MakeMatrixPacket__PKc_i(const char* matrix_data_serialized, int size, mediapipe::Packet** packet_out) {
  TRY
    mediapipe::Matrix matrix;

    // convert matrix data from char into mediapipe::MatrixData
    std::string content;
    mediapipe::MatrixData matrix_data;
    mediapipe::file::GetContents(matrix_data_serialized, &content);  // <-- using this function feels wrong!
    CHECK(matrix_data.ParseFromString(content));

    // fill matrix with data from matrix_data_serialized
    mediapipe::MatrixFromMatrixDataProto(matrix_data, &matrix);

    *packet_out = new mediapipe::Packet{mediapipe::MakePacket<mediapipe::Matrix>(matrix)};
    RETURN_CODE(MpReturnCode::Success);
  CATCH_EXCEPTION
}

As I said: it compiles, next will be testing. Meanwhile: Any feedback is welcome!
that "hack" was inspired by looking at calculator_graph.cc:
MpReturnCode mp_CalculatorGraph__PKc_i(const char* serialized_config, int size, mediapipe::CalculatorGraph** graph_out) {
  TRY_ALL
    auto config = ParseFromStringAsCalculatorGraphConfig(serialized_config, size);
    *graph_out = new mediapipe::CalculatorGraph(config);
    RETURN_CODE(MpReturnCode::Success);
  CATCH_ALL
}

me slowly moving forward. Meanwhile I've written some C# functions to paste data to the graph, but I'm still getting an Error:
MediaPipeException: INVALID_ARGUMENT: Graph has errors:
Calculator::Open() for node "TfLiteInferenceCalculator" failed: ; Can't find file: mediapipe/models/adder_model_single_input_2x3.tflite
Mediapipe.Status.AssertOk () (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/Port/Status.cs:147)
Mediapipe.Unity.HelloWorld.Start () (at Assets/MediaPipeUnity/Tutorial/Hello World/HelloWorld.cs:94)

Though it looks like the conversion of my input data didn't crash. The data itself might be corrupt though. Is there a way to debug / step into the C++ code while running Unity?

// convert matrix data from char into mediapipe::MatrixData
std::string content;
mediapipe::MatrixData matrix_data;
mediapipe::file::GetContents(matrix_data_serialized, &content);  // <-- using this function feels wrong!
CHECK(matrix_data.ParseFromString(content));

See
https://github.com/homuler/MediaPipeUnityPlugin/blob/a48603aa39ccee03d4ec158fd85778432aac9366/mediapipe_api/framework/calculator_graph.cc#L12-L13
What you should do here is convert const char* to const std::string and gives it to ParseFromString.
Although mediapipe::file::GetContents return std::string, as its name implies, it doesn't convert it to std::string but reads the contents from the given path.
matrix_data_serialized is not a file path (it is rather content), so it should always fail.
mediapipe::MatrixData matrix_data;
CHECK(matrix_data.ParseFromString(std::string(matrix_data_serialized, size)));

Is there a way to debug / step into the C++ code while running Unity?

Please google it.

Found a solution how to hit on C++ breakpoints inside the mediapipe_c.dll
That said, it seems that my code is breaking at
    std::string content;
    mediapipe::MatrixData matrix_data;
    CHECK(matrix_data.ParseFromString(std::string(matrix_data_serialized, size)));

    // fill matrix with data from matrix_data_serialized
    mediapipe::MatrixFromMatrixDataProto(matrix_data, &matrix);  // <- THIS LINE

From what I can I see so far it looks like the way I'm putting data into "MatrixData" is wrong. The parser cannot infer any matrix shape (rows, cols) from the byte string.
Will look further into this tomorrow.

data input is working now. I simply had to manually specify the shape of the input data:
var matrix = new MatrixData();
matrix.PackedData.Add(0);
matrix.PackedData.Add(1);
matrix.PackedData.Add(2);
matrix.PackedData.Add(3);
matrix.PackedData.Add(4);
matrix.PackedData.Add(5);

matrix.Rows = 2;
matrix.Cols = 3;

Next problem: My graph returns a vector<float>, which I'm currently unable to receive.
The current code is crashing Unity:
var poller = graph.AddOutputStreamPoller<float[]>("out").Value();
//...
//start graph
//...
Debug.Log("Poll output");
var output = new FloatArrayPacket();
while (poller.Next(output))  // <-- THIS LINE CAUSES UNITY CRASH
{
  Debug.Log(output.Get());
}

I'll keep on digging...

FloatArrayPacket stores float[], not std::vector<float>.
To receive std::vector<float>, you need to implement a new Packet class (FloatVectorPacket) or a calculator to convert std::vector<float> to float[].

puhh finally I got things working! 😄 🚀 🍻
thanks a lot for your help @homuler !
I will try to clean the code a bit, feel free to comment the code quality in case you are interested in me creating a pull request out of this

FYI: I rebased my changes onto the current master. How should I go forward to make a pull request out of this?

do you want to look at / give some feedback to the commits?

are variable names fine?

should I clean up the commits -> sum up everything into two commits?

one for C#
and one for C++ part)

automatic tests seem to be failing, do I need to write some tests?
what about binary files?

I have a tiny tflite model (just adds "+1" to the input matrix)
should I commit that as well to make the matrix classification demo runnable?

do you want to look at / give some feedback to the commits?

are variable names fine?

If you send me a PR, I'll review it.

should I clean up the commits

Please do it in a way that you think is best.

automatic tests seem to be failing, do I need to write some tests?

By default, tests will fail in a forked repository, so never mind.
It is preferable to write test code for the source code under Packages (e.g. https://github.com/homuler/MediaPipeUnityPlugin/blob/6efa5e1b75b7c6aa487254ade15fe7cd49fc6842/Packages/com.github.homuler.mediapipe/Tests/EditMode/Framework/Packet/BoolPacketTest.cs).

should I commit that as well to make the matrix classification demo runnable?

No, you don't need to do so.


Fix CI on TensorFlow

This PR is a follow-up to #10674. Is there a difference between build environments for PRs and the master branch? :( There was no problem in #10674 but the recent CI on the master branch have been failing.

@fchollet, I tried the following (A: data_utils_test.py, B: test_multiprocessing.py):

Enabling A, Enabling B with WORKERS=2 (success on #10674, but fail on the master)
Enabling A, Enabling B with WORKERS=1 (fail on this PR)
Enabling A, Disabling B (fail on this PR)
Disabling A, Enabling B with WORKERS=2 (fail on this PR)
Disabling A, Disabling B (success on this PR, but on the master..?)

If the problem persists after this PR is merged, we may need to revert TensorFlow 1.7 again.


Defrag datalink ppp 70x/v4

#1765 with non-ppp tests enabled for 7.0.x

⚠ This PR is failing master, I guess #1765 should be merged first when its suricata/master PR gets merged

Merged in #1772, thanks!


Feat: use redis for caching

Overview
Addresses ticket #234426
Modifies the codebase to start using the Redis cache that support was added for in a previous PR, but was not yet in use
This is intended to be the smallest amount of change possible to get it working
This won't be merged on its own but when the cleanup piece is ready as well
Changes
Major

Any calls to the database for CMS content has been removed in favour of retrieving the content from Contentful
Created a new ICmsCache inheriting from IDistributedCache to include methods specific to Contentful around invalidating the cache upon changes to Content
Removed or amended tests that no longer passed due to this change
There is still a lot of refactoring to do here to remove now redundant code, but there will be another PR for this to reduce strain on review

Non-functional changes

Made some changes to existing documentation on Contentful->DB processes and changed this to explain how we use the Redis cache to store items and now use the webhook to invalidate them
More to do here, which will come in the follow on work to cleanup

How to review the PR

Follow the instructions in the Redis Readme to connect to our Azure redis or your own
Run plan-tech, and test that the logs show it retrieves items from the cache once they've been retrieved once
Try loading the start page (for example), then updating some content within the page in Contentful, the cache should invalidate the right items and show the correct updated content when the page is refreshed.

Release requirements
Running the terraform to create the cache on test/staging/prod and setting the connection string in the keyvault for each environment
Additional notes
Checklist

[ ] Title uses Angular commit convention
[ ] PR targets development branch
[ ] Unit tests have been added/updated
[ ] Documentation has been updated where relevant
[ ] Any Key Vault secrets have been added to the development Key Vault

👍 LGTM
Only comment would be; do the various queries/commands need to use the ICmsCache interface, or would the IDistributedCache be sufficient?

I think the distributed cache would be enough, I thought I might be taking away from it as a base for a separate use of redis should there ever be one by making it too Contentful specific but I'm happy to adjust that


release update note

Signed-off-by: Kevin Lumbard<EMAIL_ADDRESS>After this PR is merged, the metric will go live on the website.

LGTM

Signed-off-By<EMAIL_ADDRESS>


added build instructions to the developing section of readme

I added build instructions to the developing section of README.md .
The last paragraph is untouched by me; the »change« seems to come by saving it.

Build instructions are in the Wiki


[help] How can I integrate latexindent  into alpheleia?

I have alias for latexindent: alias latexindent="~/tools/latexindent.pl/latexindent.pl -l=~/indentconfig.yaml"
I am able to run it as:
~/tools/latexindent.pl/latexindent.pl -l=~/indentconfig.yaml myfile.tex > output.tex

But inside emacs I am having following error related to latexindent:
*apheleia-latexindent-log*
$ latexindent --logfile\=/dev/null
Can't open /dev/null at /usr/local/texlive/2021/texmf-dist/scripts/latexindent/LatexIndent/Document.pm line 165, <> line 328.
Command failed with exit code 2.
Fri Jan 14 12:41:34 2022 :: /Users/user/folder
$ latexindent --logfile\=/dev/null
Can't open /dev/null at /usr/local/texlive/2021/texmf-dist/scripts/latexindent/LatexIndent/Document.pm line 165, <> line 328.
Command failed with exit code 2.

How can I fix this error in order to run latexindent with my given config file?

Is this on Windows or something? I'm confused why you'd be getting Can't open /dev/null.
But in any case, shell aliases only exist within the shell. They are not picked up by Emacs or by any other program that runs commands on your behalf. If you want to change the options that Apheleia passes to latexindent, you either need to edit apheleia-formatters or you need to create a wrapper script for latexindent that you put on your PATH.


Implement ORAS helper command

azoras is a tool that helps work with the ORAS CLI and Azure ACR.
The subcommands implement common workflows for image annotations and maintenance.
It currently implement three basic workflows:

azoras install: Install the ORAS CLI from GitHub releases
azoras login: Logs in the ORAS CLI to Azure ACR using the Azure CLI
azoras deprecate: Adds a end-of-life annotation to the given image or list of images

Moved to our internal repo for now while we work on a pipeline etc.


Facet entries in the discover/facets/{facet} endpoint does not return the authority key

Describe the bug
When calling the {dspace.url}/api/discover/facets/{facet} endpoint, the results won't include the authority key, even though it should be there.
To Reproduce

Set up a discovery core that has authority values for facets in the config
Try to retrieve the values through the call above
The resulting facet entries won't include the authority key

Expected behavior
The authority keys should be present in the response
Related work
This was discovered due to the failing tests in this PR: https://github.com/DSpace/DSpace/pull/3176

Thanks @Raf-atmire and @benbosman .  This does seem like something that needs fixing.  I suspect it also will require minor changes to the REST Contract (as I don't see any examples where an authority key would be returned as part of a facet): https://github.com/DSpace/RestContract/blob/main/search-endpoint.md
I'll move this over to Beta 5 as it seems small.  Is this something your team would like to work on, or does this overlap with #3177 enough that we should talk with @abollini about whether his team would be interested in implementing both together?

@tdonohue Raf can work on this as part of https://github.com/DSpace/DSpace/pull/3176
Is it ok if this implies that PR is not fixed by Thu morning (it is already an open PR)
We will of course try to get it ready in time

@benbosman : That's fine. I'll update #3176 to note that it will fix this issue.


[Backport] [2.x] Restore support for Java 8 for RestClient (#11562)

Backport of https://github.com/opensearch-project/OpenSearch/pull/11562 to 2.x

@dblock mind please approving the backport as well? thank you

❌ Gradle check result for e99fe6a: null
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

https://build.ci.opensearch.org/job/gradle-check/31154/ this check run succeeded

❌ Gradle check result for e99fe6a: FAILURE

https://github.com/opensearch-project/OpenSearch/issues/10006
https://github.com/opensearch-project/OpenSearch/issues/9191
https://github.com/opensearch-project/OpenSearch/issues/5329


Merging mhof dev with gamma_reg_collision and master

@MatteoWohlrapp , please fix #806 first, then merge #806 changes to fbopt (just realized mhof-dev has no CI attached so you never know if your edits were correct.

Codecov Report
Attention: Patch coverage is 61.71004% with 309 lines in your changes are missing coverage. Please review.

Project coverage is 90.77%. Comparing base (44271c8) to head (5b28c73).

Files
Patch %
Lines

domainlab/utils/generate_fbopt_phase_portrait.py
0.00%
199 Missing :warning:

domainlab/algos/trainers/fbopt_setpoint_ada.py
75.37%
33 Missing :warning:

domainlab/algos/msels/c_msel_val_top_k.py
29.62%
19 Missing :warning:

domainlab/algos/trainers/fbopt_mu_controller.py
84.03%
19 Missing :warning:

domainlab/models/model_diva.py
72.13%
17 Missing :warning:

domainlab/algos/builder_fbopt_dial.py
44.44%
5 Missing :warning:

domainlab/algos/trainers/train_fbopt_b.py
92.85%
5 Missing :warning:

domainlab/algos/msels/c_msel_setpoint_delay.py
84.21%
3 Missing :warning:

domainlab/algos/trainers/train_dial.py
60.00%
2 Missing :warning:

domainlab/models/model_hduva.py
88.88%
2 Missing :warning:

... and 5 more

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #820      +/-   ##
==========================================
- Coverage   95.17%   90.77%   -4.41%
==========================================
  Files         128      137       +9
  Lines        5080     5853     +773
==========================================
+ Hits         4835     5313     +478
- Misses        245      540     +295

Flag
Coverage Δ

unittests
90.77% <61.71%> (-4.41%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Do not merge][gitbase-web] use ClusterIP on port 80 by default

Change default service type from NodePort to ClusterIP, which is the
default ServiceType in Kubernetes.
Change default port from 8080 to 80 which, being the standard, is more
intuitive. There is no point in using 8080 when we have a dedicated
virtual IP.

Signed-off-by: Santiago M. Mola<EMAIL_ADDRESS>

We could apply this change to all charts where it applies, @rporres @meyskens what do you think?

This has a wider implication, as GCE ingress needs NodePort services. But I do agree that the default behaviour should be ClusterIP and port 80.
I assume that you mean only for HTTP services, isn't it? We have a bunch of gRPC services too in these charts.
Please do not merge this until changes are made in gitbase-web CD

Please do not merge this until changes are made in gitbase-web CD

I assume that you mean only for HTTP services, isn't it? We have a bunch of gRPC services too in these charts that should not follow any particular convention

@rporres Yes, I meant for HTTP, specially Web UI.


[RFC] Repo tidy ups

Wondered if you'd like to use husky with prettier and commitlint to enforce some styling of code and commits?
I didn't like prettier at first (it doesn't format code exactly like I do!) but once I got used to letting it do its thing and not worry about it, it's just nice to have everything consistently styled. That said I don't get the predilection for 80 columns hence the custom setting. Anyway, just an idea. If you'd like to use it I can update the PR to whatever prettier/commitlint configs you like. Works OK on Windows as long as git is in the path.
3rd commit is to add standard-version for changelogs and tags if the commitlint is a go-er. I'm not sure what the current release procedure is, so might not be directly applicable and there's no single script in this, but manually yarn release && yarn package would do the trick.
And then one last one to have a single resources directory - couldn't see the need for separate ones but I may be missing something!
Regards

@dcbaker @xclaesse what are your thoughts on this? I definitely think using prettier with some config would be good if we could come up with a good format. It would help in reviews. I could go for commitlint, but I am good either way.

Luke, want to split the prettier stuff into another PR and this one can be just for commitlint? That way focus can be on one thing at a time?

Sure I can give it a whirl when I get a moment.

Once the other PR is merged, do we need to add commitlint or anything else from this PR?


Error running !python /content/privateGPT/privateGPT.py

Runningn !python /content/privateGPT/privateGPT.py raises
Traceback (most recent call last):
  File "/content/privateGPT/privateGPT.py", line 89, in <module>
    main()
  File "/content/privateGPT/privateGPT.py", line 44, in main
    llm = GPT4All(model=model_path, max_tokens=model_n_ctx, backend='gptj', n_batch=model_n_batch, callbacks=callbacks, verbose=False)
  File "/usr/local/lib/python3.10/dist-packages/langchain/load/serializable.py", line 74, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Failed to retrieve model (type=value_error)

Any fix, please?

Check that the model file is present and is not corrupted.
Also, make sure you are using the correct model file
You can also try switching the model.

Thank you for your response.
I did all of the above, yet issue persists.

Is the model well-loaded?
Has the model path been properly updated in the environment file? env

Yes, it is.
I tried running it both locally and on Google Colab, but same issue persists.

This was an error from my end with the way I was passing the model's path.
It is fixed now; thanks.

you welcome


Add Hampton City Crime Data

Hampton is archived on the pilot online page but @jonahadkins began an effort to start putting together hampton crime data maps after the city police department started an open data page.
The project can be found https://github.com/Code4HR/hampton-police-open-data with the dataset in the repo and readme notes where he gets the data from. He mentioned the data goes back to 2009.
He has the data uploaded to CartoDB and he put a map on it viewable at http://code4hr.github.io/hampton-police-open-data

The pilot data was self-confessed to be a little off (I think we had some data come back for somewhere in Utah?).   We're definitely interested in the linked project.  I'll dig into it as I get time.
Definitely a good link.

I also really like the way that the crimes are defined in different colors for the severity. Another possible way for us to distinguish between major and minor crimes.

Red = Murder / Attempted Murder
Orange = Rape / Assault / Child Molestation
Yellow = Robbery / Car Jacking / Drug Distribution / Domestic Disturbance
Green = Fraud / Drug Possession
Brown = DUI / Drunk In Public / Public Intoxication
Tapping into my First 48 / Law & Order / Minority Report cop show knowledge.

crimemapping.com uses specific icons which is also neat

http://www.hampton.gov/index.aspx?nid=2930
Table format of quarterly crime info with block number and identifier

http://www.hampton.gov/index.aspx?nid=2930
Table format of quarterly crime info with block number and identifier. The table is able to be downloaded. Might be a much easier place for us to start with crime.
AND there is an API we can hit

Another option but for Newport News. We would have to slowly build up our database, but they have daily crime reports here http://www.nngov.com/744/Daily-Offense-Reports by day of the week.

I had crime data in ES, but due to the poor implementation of ES and it's miserable documentation I ended up removing the index with all that data. Working on getting that fixed again and then i'll insert the data again.

ES implementation fixed. Updated refresh-timestamp index and hercules-crime_v1 index.

created hercules-refresh_timestamp_v1 index

Data is in ES. Removed code for HRQLS code base that could duplicate data if run multiple times.


Update .gitignore to Match Yarn Recommendations for Non-Zero-Installs

This PR updates the .gitignore file to align with the recommended settings from the Yarn documentation for projects not using zero-installs.
https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored

.yarn/versions is used by the version plugin to store the package release definitions. You will want to keep it within your repository.

Because we don't use the version plugin, I'm not sure this is helpful. My local folder is basically full of junk:
It's 117 files like this:
0f0a81e8.yml
undecided:
  - root-workspace-0b6124
  - "@eventstore-ui/components"
  - "@eventstore-ui/editor"
  - "@eventstore-ui/fields"
  - "@eventstore-ui/forms"
  - "@eventstore-ui/illustrations"
  - "@eventstore-ui/layout"
  - "@eventstore-ui/router"
  - "@eventstore-ui/stores"
  - "@eventstore-ui/theme"
  - "@eventstore-ui/utils"
  - "@eventstore-ui/assets"
  - "@eventstore-ui/configs"
  - "@eventstore-ui/icon-manager"
  - "@eventstore-ui/stencil-markdown-plugin"


View outbid

not a good user exprience to go to the list of offers of each PMP to figure out is my offer is the best one or not. I should be able to view that I was out bid and I should be able to update my offer in place.

Cuong update new tab "My Bid" on User profile section please check @phoebehoang85


Build fails with "undefined: SetNumberOfThreads"

The build succeeds in my local repo, but fails in Earthly. The failing step looks like this:
+ go build -work -x -o .build/lynxkite-sphynx/lynxkite-sphynx github.com/lynxkite/lynxkite/sphynx/lynxkite-sphynx
WORK=/tmp/go-build4213354748
mkdir -p $WORK/b067/
cat >$WORK/b067/importcfg << 'EOF' # internal
# import config
packagefile os=/opt/conda/go/pkg/linux_amd64/os.a
packagefile strconv=/opt/conda/go/pkg/linux_amd64/strconv.a
packagefile runtime/cgo=/home/mambauser/.cache/go-build/9d/9d4d8aa7e2240c2e2135809074395c90183bddb1ca0fc0a1d6d6899d50aa12da-d
packagefile syscall=/opt/conda/go/pkg/linux_amd64/syscall.a
packagefile sync=/opt/conda/go/pkg/linux_amd64/sync.a
EOF
cd /home/mambauser/lk/sphynx
/opt/conda/go/pkg/tool/linux_amd64/compile -o $WORK/b067/_pkg_.a -trimpath "$WORK/b067=>" -p github.com/lynxkite/lynxkite/sphynx/networkit -lang=go1.18 -complete -buildid fsUdk08KNQ3A5X5Qu-6f/fsUdk08KNQ3A5X5Qu-6f -goversion go1.18.5 -c=4 -nolocalimports -importcfg $WORK/b067/importcfg -pack ./networkit/additions.go
# github.com/lynxkite/lynxkite/sphynx/networkit
networkit/additions.go:17:3: undefined: SetNumberOfThreads

It sounds like it could be a version issue since just started happening on main. But I've made sure go, swig, and libnetworkit all have the exact same version.
If you look closely, the generated importcfg is already wrong. It's almost empty. In the successful build importcfg is very long and includes among other things github.com/lynxkite/lynxkite/sphynx/lynxkite-sphynx and github.com/lynxkite/lynxkite/sphynx/networkit:
+ go build -work -x -o .build/lynxkite-sphynx/lynxkite-sphynx github.com/lynxkite/lynxkite/sphynx/lynxkite-sphynx
WORK=/tmp/go-build2728984828
cd /home/darabos/lk
git status --porcelain
cd /home/darabos/lk
git -c log.showsignature=false show -s --format=%H:%ct
mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile github.com/lynxkite/lynxkite/sphynx/lynxkite-sphynx=/home/darabos/.cache/go-build/f2/f2ec098b156b46c55600d1d20f07ba929498f3f7e825c6a45f726ac9f1035955-d
packagefile bufio=/home/darabos/miniconda3/envs/lk2/go/pkg/linux_amd64/bufio.a
packagefile bytes=/home/darabos/miniconda3/envs/lk2/go/pkg/linux_amd64/bytes.a
packagefile context=/home/darabos/miniconda3/envs/lk2/go/pkg/linux_amd64/context.a
packagefile encoding/json=/home/darabos/miniconda3/envs/lk2/go/pkg/linux_amd64/encoding/json.a
packagefile fmt=/home/darabos/miniconda3/envs/lk2/go/pkg/linux_amd64/fmt.a
packagefile github.com/apache/arrow/go/arrow=/home/darabos/.cache/go-build/84/8467cc5cb36571adcab7e7e45053f677b548107ba2482a796f558a3d27ef3251-d
...

I also see some git lines there, so I thought maybe the stripped environment in Earthly is to blame. I cloned the whole repo inside the Earthly environment and it's also failing. I've now created a new local Conda environment and that's failing too. So it's some package version after all, but I'm not sure which one. I'm going through the differences one by one now. 😩

I'm not sure what _go_select is, but with _go_select 2.2.0 we get this error and with _go_select 2.3.0 we don't.

Fixed in #402.


Can't mount components in version 2.0.0-rc3

Bumping @vue/test-utils from 2.0.0-rc.1 to 2.0.0-rc.3 I get that issue
vue version 3.0.5
● icon › should render the bell icon

    TypeError: 'set' on proxy: trap returned falsish for property 'hasOwnProperty'

       6 |     expect.assertions(1);
       7 |
    >  8 |     const wrapper = mount(Icon, {
         |                     ^
       9 |       props: {
      10 |         name: 'bell',
      11 |       },

      at mount (node_modules/@vue/test-utils/dist/vue-test-utils.cjs.js:2348:24)
      at Object.<anonymous> (tests/unit/specs/components/icon.spec.ts:8:21)

Hey, I'm exactly with the same issue.
Have you figured out how to fix it?


Missing file?

When I complie follow your according to your instructions, I meet the following problem:

$ make BOARD=BLUE_PILL_F103C8

Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
make: *** No rule to make target 'build-BLUE_PILL_F103C8/genhdr/pins.h', needed by 'build-BLUE_PILL_F103C8/py/mpstate.o'.  Stop.

Is there any solutions to solve this problem?

This project is not finished and does not build.

Still needs lots of work to get the F103C8 working...
In the meantime, check out the more powerful F407VET6 board from Vcc-gnd on Taobao: https://github.com/mcauser/VCC_GND_F407
Same guys that make the "blue pill".


[issue]: clock seconds is not live

 "clock": {
        "format": "{:%H:%M:%OS}",
        "rotate": ${r_deg},
        "interval": 1,
        "format-alt": "{:%R 󰃭 %d·%m·%y}",
        "tooltip-format": "<tt>{calendar}</tt>",
        "calendar": {
            "mode": "month",
            "mode-mon-col": 3,
            "on-scroll": 1,
            "on-click-right": "mode",
            "format": {
                "months": "{}",
                "weekdays": "{}",
                "today": "{}"
            }
        },
        "actions": {
            "on-click-right": "mode",
            "on-click-forward": "tz_up",
            "on-click-backward": "tz_down",
            "on-scroll-up": "shift_up",
            "on-scroll-down": "shift_down"
        }
    },

it still doesnt refresh the second hand its frustrating. i wanted it to be live like a real clock. pls tell me if its possible or not

? only %T? does that come in 24h format?

oh no wait it is updating now idk what went wrong but it just works. thanks for dealing with me


CoreIR now has a real library and include interface. Will break Jeff's code.

Jeff, please create a branch for your stuff and then verify using the coreir/libreorg branch of CGRAFlow
items that will likely need to be fixed:
Add to Makefile:
Add -lcoreir  -lcoreir-stdlib -lcoreir-passes (Make sure to add at END of command)
Add -L<Path to coreir/bin>
Add -I<Path to coreir/incldue>
change getStdlib -> CoreIRLoadLibrary_stdlib
Look at one of my tests (coreir/tests/unit/*.cpp) to see how to redo your includes for cpp/hpp files

After downloading a clean copy of coreir and moving to the libreorg branch, I'm having a problem after "make install" with "make test." Below is the error:
build/ctest: error while loading shared libraries: libcoreir-c.so: cannot open shared object file: No such file or directory

@jeffsetter, Sorry I also forgot to mention that you need to point your LD_LIBRARY_PATH to coreir/bin (or if you are on mac, point DYLD_LIBRARY_PATH)

Could we instead use -rpath? Something like this, so I don't have to add to my LD_LIBRARY_PATH?
LIBS = -Wl,-rpath=$(BIN) -lcoreir-stdlib -lcoreir-passes -lcoreir

@jeffsetter I do not know how to get this to work for both OSX and Linux. If you have any ideas on how to get it to work, can you stop by?

@jeffsetter, just pushed the update to use rpath instead

@steveri, @nikhilbhagdikar  I am unsure of what the bug is here. Could you take a look at this log? https://travis-ci.org/StanfordAHA/CGRAFlow/jobs/226615471
The changes we did ideally should not have affected anything with the cgra/testbench generator.

OMG I'm so sorry I can't believe I didn't check my mail since Thu evening!
Anyway, I'm pretty sure this is the test.py problem that I just sent mail to Caleb about (I think you (Ross) got cc'ed, yes?)
Steve
  From: rdaly525<EMAIL_ADDRESS>
To: StanfordAHA/CGRAFlow<EMAIL_ADDRESS>Cc: steveri<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Sent: Thursday, April 27, 2017 7:13 PM
Subject: Re: [StanfordAHA/CGRAFlow] CoreIR now has a real library and include interface. Will break Jeff's code. (#7)
@steveri, @nikhilbhagdikar I am unsure of what the bug is here. Could you take a look at this log? https://travis-ci.org/StanfordAHA/CGRAFlow/jobs/226615471The changes we did ideally should not have affected anything with the cgra/testbench generator.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I just reran Build #227 on travis (as referenced above) and it appears to work. Is this resolved and can be merged into master?

Closing this. Fixed by #8


DUAL-10 Fix: store tick prices and lookup instead of doing log calculation

Previously we used float64 log to calculate a tick from price. There is no strong guarantee that float64 math will perform the same on all issues and poses a small risk of being consensus-breaking. To fix this we switch to using a pre-calculated array of prices which we binary search to find the correct tick for a given price. This is also slightly more efficient and gives us the option to stop doing all price calculations in the future.
Benchmarks:
BenchmarkCalcTickIndexFromPriceV2   	  301816	      5249 ns/op
BenchmarkCalcTickIndexFromPrice      	  206276	      6662 ns/op
The price data structure is loaded into memory from a .gob file at startup. The full data structure is calculated to use about 8mb of space, so should have a negligible impact on overall performance

https://github.com/neutron-org/neutron-tests/actions/runs/9333093601

https://github.com/neutron-org/neutron-tests/actions/runs/9333093601

https://github.com/neutron-org/neutron-tests/actions/runs/9333093601


Release 1.3.6 to pypi

Added bumpversion support, and release notes. It has been uploaded to pypi

👍


Lab 5.3 - Exercise 4 process customer registration - record does not exist

Module: 5.3
Lab/Demo: 4
Task: 02
Step: c
Description of issue

the record referred to in the lab does not exist - i assume we take order 101005 - for Francematic?

Hi, Updated the lab guide accordingly for Francematic

that's great. thank you.


No software Licence

I might be able to contribute for a work project but until the software is licensed it won't be possible.
Do you think you could apply a software license to this repo?

MIT, done :)

:+1:


Influxdb query gets curl: (52) Empty reply from server

I am running a complex query on Influxdb version 1.4.2. I can see that the query is running (using show queries). But if the query reaches more than 60 seconds. I get curl: (52) Empty reply from server I tried to change query-timeout from "0" to "5m"
query-timeout = "5m"
But it did not change anything. Any suggestions ?
https://stackoverflow.com/questions/47855352/influxdb-query-gets-curl-52-empty-reply-from-server

My mistake - Load balance was the reason for this behavior.


ML 1.1 : Variables, Range, Population Distribution, Sample Distribution (A)

Description 📜
Added audio for ML 1.1 : Variables, Range, Population Distribution, Sample Distribution (A)

Type of change 📝

[x] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)

Domain of Contribution 📊

[x] Machine #Learning

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

@abhisheks008 , Updated again with correct thumbnail. Kindly review.

PR approved 🎉


Allow duration to be specified in multiple formats

For example

3.5h
3m
2 (defaults to minutes)
2h4m2s

Fixes #514

LGTM, thanks again.  And thanks for fixing those missing asserts!


No available version compatible with the installed Atom version: 0.207.0

Unable to install

Same happens with 0.208.0 and 0.209.0

seems like, there is an depricated API issue. Please fix it

Would try to get to this, in the weekend...

Nice @skandasoft! :+1: thanks

finally was able to get it done..Hopefully no more version breaking updates from ATOM...
Check the latest updates...Closing the issue..feel free to reopen/open new issues


v7.2.3 not available on Terraform Registry

Describe the bug
Version 7.2.3 is not available on Terraform Registry
To Reproduce
Steps to reproduce the behavior:

Go to https://registry.terraform.io/modules/cattle-ops/gitlab-runner/aws/latest
Observe latest version being v7.2.2

Expected behavior
v7.2.3 to be available
Additional context
/

Does the tagging style changing from 7.2.2 to v7.2.3 have anything to do with this?

Fwiw, I worked around this by pointing directly to the tag on this repo:
module "gitlab_runner" {
  source      = "github.com/cattle-ops/terraform-aws-gitlab-runner?ref=v7.2.3"

Ohhmm, that is strange and should not be the case. Every release should automatically available in the Terraform registry as well.

Seems that the problem was introduced with the update of release-please to v4. The configuration changed and I overlooked it.
https://github.com/google-github-actions/release-please-action#upgrading-from-v3-to-v4

This was not the real problem. Terraform registry supports v prefixes. I found out that the webhook which triggers the Terraform registry, doesn't work anymore. Instead the releases are sent to npalm/gitlab-runner which makes no sense. Have to reach out  @npalm to get this fixed.

All releases are indeed available at https://registry.terraform.io/modules/npalm/gitlab-runner/aws/latest. Maybe some URL fall-back after rename expired.

Hi @npalm could you please resync the Terraform module in the registry to setup a new GitHub webhook which hopefully works again?
https://registry.terraform.io/modules/cattle-ops/gitlab-runner/aws/latest

Will check tonight

Resynced the module

7.4.0 has the same issue, still not available on Terraform after 4 days of it being released.

@npalm Could you please check again?
Repository settings are fine. The webhook is still there and marked as active.

Juat re-synced the module in the registry (manually). No clue why it did not synced. For another module I had the problem as well some weeks ago. There we release including the v prefix. But last weeks no issues.

I still can't see 7.4.0 in the Terraform registry

I still can't see 7.4.0 in the Terraform registry

You are probably looking at old npaln's registry, this is the proper one: https://registry.terraform.io/modules/cattle-ops/gitlab-runner/aws/latest

@jonpas that makes sense. Thanks for pointing that out!

7.4.0 is in the registry


chore: update DevPod

https://github.com/loft-sh/devpod/releases/tag/v0.5.18

Sorry about the commit message!


Release 3.23.0

New Analytics and custom events.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

111 of 205 (54.15%)  changed or added relevant lines in 54 files are covered.
24 unchanged lines in 13 files lost coverage.
Overall coverage increased (+0.4%) to 35.516%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/components/AppLayout/Header/components/NetworkSelector.tsx
0
1
0.0%

src/components/CookiesBanner/index.tsx
0
1
0.0%

src/components/SafeListSidebar/index.tsx
0
1
0.0%

src/logic/wallets/onboard.ts
0
1
0.0%

src/routes/LoadSafePage/steps/LoadSafeAddressStep.tsx
4
5
80.0%

src/routes/index.tsx
0
1
0.0%

src/routes/safe/components/AddressBook/EllipsisTransactionDetails/index.tsx
0
1
0.0%

src/routes/safe/components/AddressBook/ImportEntriesModal/index.tsx
0
1
0.0%

src/routes/safe/components/Apps/components/AddAppForm/index.tsx
0
1
0.0%

src/routes/safe/components/Balances/SendModal/screens/ReviewSendFundsTx/index.tsx
0
1
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/components/SafeListSidebar/index.tsx
1
0%

src/components/SafeListSidebar/SafeList/index.tsx
1
0%

src/routes/safe/components/AddressBook/EllipsisTransactionDetails/index.tsx
1
10.0%

src/routes/safe/components/AddressBook/index.tsx
1
0%

src/routes/safe/components/Apps/components/AppFrame.tsx
1
0%

src/routes/safe/components/Balances/SendModal/screens/ReviewSendFundsTx/index.tsx
1
0%

src/routes/safe/components/Settings/ThresholdSettings/ChangeThreshold/index.tsx
1
0%

src/routes/safe/components/Settings/Appearance/index.tsx
2
0%

src/routes/safe/components/Settings/SafeDetails/index.tsx
2
0%

src/routes/safe/components/Transactions/TxList/hooks/useActionButtonsHandlers.ts
2
1.82%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
3465

Relevant Lines:
8832

💛  - Coveralls


Error: You must first run "eb init".

I get this error when I try to deploy, and I do have my .elasticbeanstalk/config.yml committed into the repo.

I'm sure you checked this, but just in case make sure that your .gitignore is not preventting .elasticbeanstalk/config.yml from being uploaded. EB automatically adds stuff to your gitignore which prevents upload. That was the problem for me.

Interesting that we would need to .gitignore the config, or worry about the config when this should include eb init. We are already bringing in the secrets into the action. Just curious as to why we couldn't include the eb init script in the action, maybe in commands with deploy?

Also make sure that you use actions/checkout@v2, so that the actual .elasticbeanstalk/config.yml file can be read from the commit.
Ex:
jobs:
  update_environment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses<EMAIL_ADDRESS>        with:
          command: 'setenv BUNDLE_URL=${{ secrets.BUNDLE_URL }}'
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_DEFAULT_REGION: "us-west-2"


PHOENIX-7172: Support HBase 2.6

Remove files left from Hbase 2.3 compat module

I removed the HBase 2.3 compat module as it should have been removed in PHOENIX-6877
And added the new 2.6 modules, git recognises these as renamed some files which is not ideal, maybe it should be done separately.
Also this change can only be merged after the 2.6 release

We still have time, I agree that it'd be cleaner if we removed any 2.3 leftovers in a separate ticket.

Please update the commit message, @richardantal

Thank you.
Let's get get back to this once 2.6.0 is out.

Now that HBase 2.6.0 is released, we can resume here?

Kicked off a new CI build

Can you rebase @richardantal ? The patch doesn't apply cleanly anymore.

It is based on an old version, from before we removed 2.4.0. support from HEAD.

Also this still uses 2.6.0-SNAPSHOT.
Updating to 2.6.0-hadoop3 should let the CI run fully.

Also update the Phoenix version to 5.3.0-SNAPSHOT in the new compatibility module, I've spent an hour trying to compile it.

CompatOmidTransactionTable also needs to be updated.

I see that hose issues are fixed.
Sorry, I must have used an older patch.

However, most of the checkstyle warnings are relevant.
I'm not sure why the javadoc check failed.

Does this work fine locally?
mvn clean javadoc:javadoc -Dskip.code-coverage  -DskipTests=true

Yes, it works after mvn install.
Yetus should have taken care of that.

Looks like we need to add support to run tests against 2.6 profile https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/688/ ?

Yes, opened https://issues.apache.org/jira/browse/PHOENIX-7333 for that.

Thanks @stoty @richardantal for this great work!
Reg the precommit jenkins builds, i just noticed that 2.6 builds are getting aborted (whereas 2.4 and 2.5 builds get failed with few test failures but at least the builds are not aborted).

You mean the multibranch jenkis job, @virajjasani  ?
We all precommit builds are running with 2.5 only at the moment.

My bad, yes multibranch builds of master and 5.2

Opened https://issues.apache.org/jira/browse/PHOENIX-7339

Thanks a lot @stoty @richardantal for getting HBase 2.6 support in and stabilizing tests.
It seems like one last thing we might have to do is adjust client version for 2.6 profile? https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/703/testReport/junit/org.apache.phoenix.end2end/BackwardCompatibilityIT/MatrixBuild___Matrix___HBASE_PROFILE____2_6____BuildAndTest___initializationError/


Adding test hook for UI automation

@shoatman Can you share a link to the tests where this is actually being used?

They are in OneAuth repo: https://office.visualstudio.com/OneAuth/_git/OneAuth?version=GBandroid%2Foneauth-msal-uiautomation-tests&path=%2Ftestapps%2Fandroid%2FOneAuthTestApp%2Fapp%2Fsrc%2FandroidTest%2Fjava%2Fcom%2Fmicrosoft%2Fauthentication%2Ftestapp%2FWaitForWebViewToLoad.java
https://office.visualstudio.com/OneAuth/_git/OneAuth?path=%2Ftestapps%2Fandroid%2FOneAuthTestApp%2Fapp%2Fsrc%2FandroidTest%2Fjava%2Fcom%2Fmicrosoft%2Fauthentication%2Ftestapp%2FSignInTests.java&version=GBandroid%2Foneauth-msal-uiautomation-tests&line=109&lineEnd=120&lineStartColumn=1&lineEndColumn=1&lineStyle=plain


[Bug] 同批json格式数据在1.1.x版本无法正常写入doris

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
flink-doris-connector-1.14_2.12:1.1.0
flink.version:1.14.5
doris:1.2.2
What's Wrong?
datastream的侧输出流无法正常写入doris，格式跟主流是一致的json格式。
主流是正常写入的，而且将doris connector换成1.0.3后却可以正常写入
这部分数据我写到本地用文件流的方式去写入测试也是一样，1.1版本无法写入，但是也没有任何报错信息，
日志显示了正常的写入条数，且success。
What You Expected?
1.1版本对flink侧流是有bug吗，但是你们官方指定1.14以上的flink需要用1.1以上的连接器，这个问题能否解决一下
How to Reproduce?
No response
Anything Else?
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Whether checkpoint is configured ?

Whether checkpoint is configured ?

Yes, I configured the checkpoint to hdfs.

You need configured the checkpoint to flink.

You need configured the checkpoint to flink.

You meaned I should configured the checkpoint to flink job maneger notly hdfs?
If the job failed, lost all of the data?
My question is,
The same data, I can sink doris by doris connector version 1.0.3,but 1.1.0 cannot,my flink version is 1.14.5,
on the doris connector document website, flink1.14.x depend on doris connector 1.1.x.
What does it have to do with checkpoints?

This is my wechat: smallhibiscus, you can contact me in private.


Add spin integration

See https://github.com/fermyon/spin and https://github.com/fermyon/leptos-spin

Needs https://github.com/fermyon/leptos-spin/issues/42


ci: disable packaging and building workflows on draft PRs

These workflows fail on draft PRs anyway, and I believe they will get run when the draft status is removed.
I still think it's important to have the precommit workflow run, but these others don't seem too important for a draft.

I was wrong about the packaging ones failing, this was because the Cargo.lock was out of line with master, but I still don't think these need to run. Feel free to close if you disagree @elijah-potter, I don't have a strong opinion on this.


fix(form-field): unable to tap on certain types of inputs on iOS

Fixes an issue that caused certain input types to collapse into 1px tall strips, preventing users from being to tap on them on iOS.
Fixes #8001.

@crisbeto please rebase. Thanks!

Rebased @tinayuangao.


bazel-watcher: init at 4d5928e

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

/cc @mboes @Profpatsch @xeji

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/cp2p2x3h8n66y3v7rl9kjy32z6ijix6d-bazel-watcher-2018-09-09
shrinking /nix/store/cp2p2x3h8n66y3v7rl9kjy32z6ijix6d-bazel-watcher-2018-09-09/bin/ibazel
cannot find section .dynamic
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/cp2p2x3h8n66y3v7rl9kjy32z6ijix6d-bazel-watcher-2018-09-09/bin
patching script interpreter paths in /nix/store/cp2p2x3h8n66y3v7rl9kjy32z6ijix6d-bazel-watcher-2018-09-09
checking for references to /build in /nix/store/cp2p2x3h8n66y3v7rl9kjy32z6ijix6d-bazel-watcher-2018-09-09...
cannot find section .dynamic
/nix/store/cp2p2x3h8n66y3v7rl9kjy32z6ijix6d-bazel-watcher-2018-09-09

Success on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/vh94yivcacz3wjx3hcziyqfn2vv81gk5-bazel-watcher-2018-09-09
shrinking /nix/store/vh94yivcacz3wjx3hcziyqfn2vv81gk5-bazel-watcher-2018-09-09/bin/ibazel
cannot find section .dynamic
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/vh94yivcacz3wjx3hcziyqfn2vv81gk5-bazel-watcher-2018-09-09/bin
patching script interpreter paths in /nix/store/vh94yivcacz3wjx3hcziyqfn2vv81gk5-bazel-watcher-2018-09-09
checking for references to /build in /nix/store/vh94yivcacz3wjx3hcziyqfn2vv81gk5-bazel-watcher-2018-09-09...
cannot find section .dynamic
/nix/store/vh94yivcacz3wjx3hcziyqfn2vv81gk5-bazel-watcher-2018-09-09

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

@srhb I was pointing to a specific revision because of my patch upstream https://github.com/bazelbuild/bazel-watcher/commit/4d5928eee3dd5843a1b55136d914b78fef7f25d0 that's needed for a successful packaging. But since my commit is just one after the released version v0.5.0, I have updated the PR to use the release and just import the patch.
PTAL.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Unexpected error: command failed with exit code 1 on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
Building: no action
Building: no action
Building: no action
INFO: SHA256 (https://codeload.github.com/google/protobuf/zip/48cb18e5c419ddd23d9badcfe4e9df7bde1979b2) = b6b42f90c60b54732f764ae875623a9b05e6eede064173c36c6fea12dd376cdd
Building: no action
Building: no action
Building: no action
installing
warning: path '/nix/store/csw5flq15rja43igbxqwmcvf5f0bxpbk-bazel-watcher-0.5.0-deps' claims to be content-addressed but isn't
error: unexpected end-of-file

@srhb not sure why but it seems to fail and succeed randamly. See #46864 for my attempt at understanding the failure.

@srhb sorry I gave you the wrong PR, I meant this PR comment https://github.com/NixOS/nixpkgs/pull/46509#issuecomment-420453352

@GrahamcOfBorg build bazel-watcher

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Unexpected error: command failed with exit code 1 on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
Building: no action
Building: no action
Building: no action
INFO: SHA256 (https://codeload.github.com/google/protobuf/zip/48cb18e5c419ddd23d9badcfe4e9df7bde1979b2) = b6b42f90c60b54732f764ae875623a9b05e6eede064173c36c6fea12dd376cdd
Building: no action
Building: no action
Building: no action
installing
warning: path '/nix/store/csw5flq15rja43igbxqwmcvf5f0bxpbk-bazel-watcher-0.5.0-deps' claims to be content-addressed but isn't
error: unexpected end-of-file

@GrahamcOfBorg build bazel-watcher

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Unexpected error: command failed with exit code 1 on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
Building: no action
Building: no action
Building: no action
INFO: SHA256 (https://codeload.github.com/google/protobuf/zip/48cb18e5c419ddd23d9badcfe4e9df7bde1979b2) = b6b42f90c60b54732f764ae875623a9b05e6eede064173c36c6fea12dd376cdd
Building: no action
Building: no action
Building: no action
installing
warning: path '/nix/store/csw5flq15rja43igbxqwmcvf5f0bxpbk-bazel-watcher-0.5.0-deps' claims to be content-addressed but isn't
error: unexpected end-of-file

@grahamc Do you know what this error is about? It appears to work just fine locally, sandboxed and all. https://logs.nix.ci/?key=nixos/nixpkgs.47448&attempt_id=4a8c1c59-ccb8-424a-b8ed-ce5d6c8fe945

@grahamc also I tried restarting the build a few times in #46509 and it did end up working once. I'll give it another try.
@GrahamcOfBorg build bazel-watcher

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: bazel-watcher
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Unexpected error: command failed with exit code 1 on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
warning: unable to download 'https://cache.nixos.org/dcagcm4bpzh3np0kaginayq7jdnbvli4.narinfo': HTTP error 503 (curl error: No error); retrying in 4622 ms
warning: unable to download 'https://cache.nixos.org/ypskrykzislxa3a4sx7pav1gkdv2jzhl.narinfo': HTTP error 503 (curl error: No error); retrying in 5139 ms
warning: unable to download 'https://cache.nixos.org/s0q9fdiyrv47invk6qkmiry2nf2pxn0b.narinfo': HTTP error 503 (curl error: No error); retrying in 18619 ms
warning: unable to download 'https://cache.nixos.org/dcagcm4bpzh3np0kaginayq7jdnbvli4.narinfo': HTTP error 503 (curl error: No error); retrying in 8548 ms
warning: unable to download 'https://cache.nixos.org/hfnqpkqjq80jyag478wcj1msqwzcgpdy.narinfo': HTTP error 503 (curl error: No error); retrying in 9070 ms
warning: unable to download 'https://cache.nixos.org/ypskrykzislxa3a4sx7pav1gkdv2jzhl.narinfo': HTTP error 503 (curl error: No error); retrying in 9215 ms
warning: unable to download 'https://cache.nixos.org/dcagcm4bpzh3np0kaginayq7jdnbvli4.narinfo': HTTP error 503 (curl error: No error); retrying in 17305 ms
warning: unable to download 'https://cache.nixos.org/hfnqpkqjq80jyag478wcj1msqwzcgpdy.narinfo': HTTP error 503 (curl error: No error); retrying in 19531 ms
warning: unable to download 'https://cache.nixos.org/ypskrykzislxa3a4sx7pav1gkdv2jzhl.narinfo': HTTP error 503 (curl error: No error); retrying in 17982 ms
error: unexpected end-of-file

Unexpected error: command failed with exit code 1 on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
Building: no action
Building: no action
INFO: SHA256 (https://codeload.github.com/google/protobuf/zip/48cb18e5c419ddd23d9badcfe4e9df7bde1979b2) = b6b42f90c60b54732f764ae875623a9b05e6eede064173c36c6fea12dd376cdd
Building: no action
Building: no action
Building: no action
installing
fixed-output derivation produced path '/nix/store/bp7540dm0hyy5cq8v2ffqidzy4kc4r4g-bazel-watcher-0.5.0-deps' with sha256 hash '1z25mgicbpiyc67xq8m93ysjrnh7f1nak4mwcxh5wf155imbxqxp' instead of the expected hash '1iyjvibvlwg980p7nizr6x5v31dyp4a344f0xn839x393583k59d'
cannot build derivation '/nix/store/ykcxdffb7s97lqqjrr7rl24ib2gbm2j9-bazel-watcher-0.5.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/ykcxdffb7s97lqqjrr7rl24ib2gbm2j9-bazel-watcher-0.5.0.drv' failed

Unexpected error: command failed with exit code 1 on x86_64-linux (full log)
Attempted: bazel-watcher
Partial log (click to expand)
Building: no action
Building: no action
Building: no action
INFO: SHA256 (https://codeload.github.com/google/protobuf/zip/48cb18e5c419ddd23d9badcfe4e9df7bde1979b2) = b6b42f90c60b54732f764ae875623a9b05e6eede064173c36c6fea12dd376cdd
Building: no action
Building: no action
Building: no action
installing
warning: path '/nix/store/bp7540dm0hyy5cq8v2ffqidzy4kc4r4g-bazel-watcher-0.5.0-deps' claims to be content-addressed but isn't
error: unexpected end-of-file

The error was traced to a bug specific to nix 2.0.1. 2.0.2 or later are fine.


Add reconnect logic to sync.Start

The API surface looks polluted, should probably separate out the Start from the Reconnect cases altogether instead of sharing code...  but will sleep on it for now.

Codecov Report

Merging #107 into master will increase coverage by 0.09%.
The diff coverage is 95%.

@@            Coverage Diff             @@
##           master     #107      +/-   ##
==========================================
+ Coverage   87.88%   87.97%   +0.09%
==========================================
  Files          19       19
  Lines        1238     1264      +26
==========================================
+ Hits         1088     1112      +24
- Misses         99      100       +1
- Partials       51       52       +1

Flag
Coverage Δ

#dot
87.97% <95%> (+0.09%)
:arrow_up:

#dot_data
65.78% <ø> (ø)
:arrow_up:

#dot_sync
84.86% <95%> (+0.62%)
:arrow_up:

#dot_undo
100% <ø> (ø)
:arrow_up:

Impacted Files
Coverage Δ

sync/client_log.go
81.81% <100%> (+0.48%)
:arrow_up:

sync/reconciler.go
89.16% <94.11%> (+0.62%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3793784...1d56995. Read the comment docs.


query has "2 hour gap" data problem despite endpoints correctly configured

Relates to https://github.com/thanos-io/thanos/issues/1435

Thanos, Prometheus and Golang version used:
thanos: 0.28.0
prometheus: v2.38.0

Object Storage Provider: S3
What happened:
I deployed thanos, grafana and prometheus via well-configured charts, and querying for metrics has a 2 hour gap.
What you expected to happen:
There should be no gap.
How to reproduce it (as minimally and precisely as possible):
The key detail is that thanos query is correctly started with:
      query
      --log.level=info
      --log.format=logfmt
      --grpc-address=<IP_ADDRESS>:10901
      --http-address=<IP_ADDRESS>:10902
      --query.replica-label=replica
      --endpoint=dnssrv+_grpc._tcp.kube-prometheus-stack-thanos-discovery.monitoring.svc.cluster.local
      --endpoint=dnssrv+_grpc._tcp.thanos-storegateway.monitoring.svc.cluster.local
      --query.timeout=5m
      --query.lookback-delta=15m
      --query.replica-label=rule_replica

Observe this has the kube-prometheus-stack sidecar as a store and thanos store as a store. Query Frontend appears to be correctly configured too:
      query-frontend
      --log.level=info
      --log.format=logfmt
      --http-address=<IP_ADDRESS>:10902
      --query-frontend.downstream-url=http://thanos-query:9090
      --query-frontend.compress-responses
      --query-range.split-interval=12h
      --labels.split-interval=12h
      --query-range.max-retries-per-request=10
      --labels.max-retries-per-request=10
      --query-frontend.log-queries-longer-than=10s

Finally, grafana is configured to use the query-frontend as a datasource. From a values.yaml for grafana in the kube-prometheus-stack chart:
grafana:
  nodeSelector:
    kubernetes.io/os: linux
  sidecar:
    datasources:
      defaultDatasourceEnabled: false
    dashboards:
      searchNamespace: ALL
  additionalDataSources:
    - name: Prometheus
      access: proxy
      editable: false
      orgId: 1
      type: prometheus
      url: "http://thanos-query-frontend:9090"
      version: 1
      isDefault: true

Name:              thanos-query-frontend
...
Port:              http  9090/TCP
TargetPort:        http/TCP
Endpoints:         <IP_ADDRESS>:10902

Full logs to relevant components:
Observe the thanos-query logs look normal. It has registered both stores.
Logs

level=info ts=2022-10-05T16:54:17.591023798Z caller=options.go:26 protocol=gRPC msg="disabled TLS, key and cert must be set to enable"
level=info ts=2022-10-05T16:54:17.591425835Z caller=query.go:724 msg="starting query node"
level=info ts=2022-10-05T16:54:17.59169538Z caller=intrumentation.go:75 msg="changing probe status" status=healthy
level=info ts=2022-10-05T16:54:17.591725401Z caller=http.go:73 service=http/server component=query msg="listening for requests and metrics" address=<IP_ADDRESS>:10902
level=info ts=2022-10-05T16:54:17.591804852Z caller=intrumentation.go:56 msg="changing probe status" status=ready
level=info ts=2022-10-05T16:54:17.592301321Z caller=tls_config.go:195 service=http/server component=query msg="TLS is disabled." http2=false
level=info ts=2022-10-05T16:54:17.592408763Z caller=grpc.go:131 service=gRPC/server component=query msg="listening for serving gRPC" address=<IP_ADDRESS>:10901
level=warn ts=2022-10-05T16:54:27.594778438Z caller=endpointset.go:416 component=endpointset msg="update of endpoint failed" err="getting metadata: fallback fetching info from <IP_ADDRESS>:10901: rpc error: code = DeadlineExceeded desc = context deadline exceeded" address=<IP_ADDRESS>:10901
level=info ts=2022-10-05T16:54:27.59487497Z caller=endpointset.go:381 component=endpointset msg="adding new sidecar with [storeAPI rulesAPI exemplarsAPI targetsAPI MetricMetadataAPI]" address=<IP_ADDRESS>:10901 extLset="{prometheus=\"monitoring/kube-prometheus-stack-prometheus\", prometheus_replica=\"prometheus-kube-prometheus-stack-prometheus-0\"}"
level=warn ts=2022-10-05T16:54:32.596079243Z caller=endpointset.go:416 component=endpointset msg="update of endpoint failed" err="getting metadata: fallback fetching info from <IP_ADDRESS>:10901: rpc error: code = DeadlineExceeded desc = context deadline exceeded" address=<IP_ADDRESS>:10901
level=warn ts=2022-10-05T16:54:37.596733662Z caller=endpointset.go:416 component=endpointset msg="update of endpoint failed" err="getting metadata: fallback fetching info from <IP_ADDRESS>:10901: rpc error: code = DeadlineExceeded desc = context deadline exceeded" address=<IP_ADDRESS>:10901
level=warn ts=2022-10-05T16:54:42.597552892Z caller=endpointset.go:416 component=endpointset msg="update of endpoint failed" err="getting metadata: fallback fetching info from <IP_ADDRESS>:10901: rpc error: code = DeadlineExceeded desc = context deadline exceeded" address=<IP_ADDRESS>:10901
level=warn ts=2022-10-05T16:54:47.598577876Z caller=endpointset.go:416 component=endpointset msg="update of endpoint failed" err="getting metadata: fallback fetching info from <IP_ADDRESS>:10901: rpc error: code = DeadlineExceeded desc = context deadline exceeded" address=<IP_ADDRESS>:10901
level=warn ts=2022-10-05T16:54:52.599565411Z caller=endpointset.go:416 component=endpointset msg="update of endpoint failed" err="getting metadata: fallback fetching info from <IP_ADDRESS>:10901: rpc error: code = DeadlineExceeded desc = context deadline exceeded" address=<IP_ADDRESS>:10901
level=info ts=2022-10-05T16:54:57.384725739Z caller=endpointset.go:381 component=endpointset msg="adding new store with [storeAPI]" address=<IP_ADDRESS>:10901 extLset="{prometheus=\"monitoring/kube-prometheus-stack-prometheus\", prometheus_replica=\"prometheus-kube-prometheus-stack-prometheus-0\"}"
level=warn ts=2022-10-05T19:41:47.30835703Z caller=proxy.go:362 component=proxy request="min_time:1664911560000 max_time:1664927940000 matchers:<name:\"__name__\" value:\"some_query\" > aggregates:COUNT aggregates:SUM step:60000 " err="No StoreAPIs matched for this query" stores="store Addr: <IP_ADDRESS>:10901 LabelSets: {prometheus=\"monitoring/kube-prometheus-stack-prometheus\", prometheus_replica=\"prometheus-kube-prometheus-stack-prometheus-0\"} Mint:<PHONE_NUMBER>000 Maxt:<PHONE_NUMBER>854775807 filtered out: does not have data within this time period: [1664911560000,1664927940000]. Store time ranges: [1664949600000,9223372036854775807];store Addr: <IP_ADDRESS>:10901 LabelSets: {prometheus=\"monitoring/kube-prometheus-stack-prometheus\", prometheus_replica=\"prometheus-kube-prometheus-stack-prometheus-0\"} Mint:<PHONE_NUMBER>000 Maxt:<PHONE_NUMBER>000 filtered out: does not have data within this time period: [1664911560000,1664927940000]. Store time ranges: [1664949600000,1664992800000]"

Anything else we need to know:
This 2 hour gap thing where it isn't getting the latest data keeps happening to people.

Red herring due to weird prometheus configuration error.


Make stream name in stream settings panel a link

To make it easier to navigate to a stream, we should try making the stream name at the of the stream settings panel be a link to that stream (same functionality as the eye button).
Notes:

For a first pass, let's not change the visuals of how the stream name is displayed. We can try it out and see if it's working or not.
Let's add a tooltip that says "View stream [Shift][V]", with the current nice Tippy tooltip styling for keyboard shortcuts.
We can keep the eye button as-is for now.

Inspired by #24941.

@Ddharmani3 Just to check, is updating the tooltip on the eye button in one of your open PRs?

@alya changes mentioned in issue description are for stream name in box 1 or for stream name in box 2?

@Ddharmani3 Just to check, is updating the tooltip on the eye button in one of your open PRs?

Yes, the tooltip on the view stream eye button is updated to tippy in #25047.

@alya changes mentioned in issue description are for stream name in box 1 or for stream name in box 2?

Box 1. Thanks!


It's too easy to show the game library

Currently the game library is shown by tapping on the Game Boy screen. A number of users have found that it's too easy to do this by mistake.
We should explore alternative mechanisms for showing the game library.

So far, the users experiencing problems have been using the following devices:

iPhone 5

This has been addressed in 2.2.0 by adding a navigation bar with a back button. On the iPhone 4, this navigation bar auto-hides.


Update sqlite3 for node 6

Any problem with your unit test ? didn't change anything but they crash

The culprit is here: https://github.com/waterlinejs/sqlite3-adapter/blob/master/package.json#L37
That should just point to the waterline-adapter-tests that's published to npm

@tjwebb here is the update to fix sqlite3 on node6 :)


fix: make sure headers must be object type

Description

According to specification message headers must be defined as a schema of an object. Other types are not allowed.
Solution based on https://json-schema.org/understanding-json-schema/reference/combining.html#id5

now validation fails with:
"headers": {
    "type": "integer",
    "enum": [1, 2]
}

but it is allowing:
"headers": {
    "type": "object",
    "properties": {
      "my-app-header": {
        "type": "integer",
        "minimum": 0,
        "maximum": 100
      }
    }
  }

Related issue(s)
See also https://github.com/asyncapi/parser-js/issues/68
Comment
I assume that once this is merged, I also need to create a PR to asyncapi-node. See also https://github.com/asyncapi/asyncapi/issues/395

@fmvilas good point, const is better, have a look now

It is actually not included in the release. This was an unintentional release caused by the release automation initial run. 1.0 version of the spec was released a long time ago. The release mentioned by the bot was removed.


TimeOut for Tokensregex service

I'm testing the Python code given in the README of the package  for the "Annotation Server Usage" - only changed the annotators back to the default ones, on line 9:
with corenlp.CoreNLPClient() as client:
  ann = client.annotate(text)

but I'm experiencing a timeout error when the programme reaches the tokensregex step:
python test.py
[main] INFO CoreNLP - --- StanfordCoreNLPServer#main() called ---
[main] INFO CoreNLP - setting default constituency parser
[main] INFO CoreNLP - warning: cannot find edu/stanford/nlp/models/srparser/englishSR.ser.gz
[main] INFO CoreNLP - using: edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz instead
[main] INFO CoreNLP - to use shift reduce parser download English models jar from:
[main] INFO CoreNLP - http://stanfordnlp.github.io/CoreNLP/download.html
[main] INFO CoreNLP -     Threads: 4
[main] INFO CoreNLP - Starting server...
[main] INFO CoreNLP - StanfordCoreNLPServer listening at /0:0:0:0:0:0:0:0:9000
[pool-1-thread-3] INFO CoreNLP - [/<IP_ADDRESS>:47736] API call w/annotators tokenize,ssplit,pos,depparse,lemma,ner
Chris wrote a simple sentence that he parsed with Stanford CoreNLP.
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.StanfordCoreNLP - Adding annotator tokenize
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.TokenizerAnnotator - No tokenizer type provided. Defaulting to PTBTokenizer.
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.StanfordCoreNLP - Adding annotator ssplit
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.StanfordCoreNLP - Adding annotator pos
[pool-1-thread-3] INFO edu.stanford.nlp.tagger.maxent.MaxentTagger - Loading POS tagger from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [0.8 sec].
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.StanfordCoreNLP - Adding annotator depparse
[pool-1-thread-3] INFO edu.stanford.nlp.parser.nndep.DependencyParser - Loading depparse model file: edu/stanford/nlp/models/parser/nndep/english_UD.gz ...
[pool-1-thread-3] INFO edu.stanford.nlp.parser.nndep.Classifier - PreComputed 99996, Elapsed Time: 11.114 (s)
[pool-1-thread-3] INFO edu.stanford.nlp.parser.nndep.DependencyParser - Initializing dependency parser ... done [12.6 sec].
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.StanfordCoreNLP - Adding annotator lemma
[pool-1-thread-3] INFO edu.stanford.nlp.pipeline.StanfordCoreNLP - Adding annotator ner
[pool-1-thread-3] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [1.1 sec].
[pool-1-thread-3] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.muc.7class.distsim.crf.ser.gz ... done [0.7 sec].
[pool-1-thread-3] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... done [0.4 sec].
[pool-1-thread-3] INFO edu.stanford.nlp.time.JollyDayHolidays - Initializing JollyDayHoliday for SUTime from classpath edu/stanford/nlp/models/sutime/jollyday/Holidays_sutime.xml as sutime.binder.1.

Traceback (most recent call last):
  File "test.py", line 32, in <module>
    matches = client.tokensregex(text, pattern)
  File "/home/cecile/workspace/python-stanford-corenlp/corenlp/client.py", line 205, in tokensregex
    matches = self.__regex('/tokensregex', text, pattern, filter)
  File "/home/cecile/workspace/python-stanford-corenlp/corenlp/client.py", line 228, in __regex
    self.ensure_alive()
  File "/home/cecile/workspace/python-stanford-corenlp/corenlp/client.py", line 107, in ensure_alive
    raise PermanentlyFailedException("Timed out waiting for service to come alive.")
corenlp.client.PermanentlyFailedException: Timed out waiting for service to come alive.

I see in client.py that there is a hack for this in the definition of __regex, but it doesn't seem to solve the problem.
def __regex(self, path, text, pattern, filter):

        """Send a regex-related request to the CoreNLP server.
        :param (str | unicode) path: the path for the regex endpoint
        :param text: raw text for the CoreNLPServer to apply the regex
        :param (str | unicode) pattern: regex pattern
        :param (bool) filter: option to filter sentences that contain matches, if false returns matches
        :return: request result
        """
        self.ensure_alive()

        # HACK: For some stupid reason, CoreNLPServer will timeout if we
        # need to annotate something from scratch. So, we need to call
        # this to ensure that the _regex call doesn't timeout.
        self.annotate(text)

(I also tried by removing the self.annotate(text) but the error is the same)
The previous steps look fine nonetheless:
sentence: token {
  word: "Chris"
  pos: "NNP"
  value: "Chris"
  before: ""
  after: " "
  originalText: "Chris"
  ner: "PERSON"
  lemma: "Chris"
  beginChar: 0
  endChar: 5
  tokenBeginIndex: 0
  tokenEndIndex: 1
  hasXmlContext: false
}
token {
  word: "wrote"
  pos: "VBD"
[...]

@dan-zheng

I found that there is an inner timeout limit of 15s in function CoreNLPClient.ensure_alive, so changing RobustService.TIMEOUT to a bigger value than 15 will have a chance to avoid timeout.

It seems that the Java CoreNLP Server is launched without memory configuration

start_cmd = "java -cp '{corenlp_home}/*' edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port {port} -timeout {timeout}".format(
                corenlp_home=os.getenv("CORENLP_HOME"),
                port=port,
                timeout=timeout)

so I add a -mx4g to allocate a 4g memory space like:
start_cmd = "java -cp '{corenlp_home}/*'  -mx4g edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port {port} -timeout {timeout}".format(
                corenlp_home=os.getenv("CORENLP_HOME"),
                port=port,
                timeout=timeout)

Don't know if these helps.

Allocating a 4g memory space for the launch of the Java CoreNLP Server did solve the problem!
Thanks!

I am trying to install and use this lib but get the same error.
Unfortunately, neither the 4g trick nor the timeout solve the issue.

same here - adding -mx4g doesn't solve the problem.

-mx4g in this term 4g is not RAM, it is the heap size that the java jdk can take to during the execution time. You can increase it to 100g even if the server shutdown's while doing this then try either of these

uninstall java jdk and install a new version iof jdk
check the available harddisk size

and then increase the tmeout to 75000 or more.
use this command to start the server
java -Xmx10g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000 -timeout 75000 -serverProperties server.properties -preload "tokenize,ssplit,pos,lemma,ner,regexner,tokensregex"
this will take a bit time to start the server,around 30-45 seconds and try to reduce the number of annotators and sentences.
try by increasing the timeout and heap size if u till face errors.

Reboot the system just solve the problem. It's weird

I still meet this issue, but in a different situation. With setting be_quiet=False, I printed the details:

[main] INFO CoreNLP - --- StanfordCoreNLPServer#main() called ---
....
[main] INFO CoreNLP - Starting server...
[main] INFO CoreNLP - StanfordCoreNLPServer listening at /0:0:0:0:0:0:0:0:3220

After waiting, I got the following error with Traceback:

Traceback (most recent call last):
doc = client.annotate(text)
r = self._request(text.encode('utf-8'), request_properties, **kwargs)
self.ensure_alive()
raise PermanentlyFailedException("Timed out waiting for service to come alive.")
stanfordnlp.server.client.PermanentlyFailedException: Timed out waiting for service to come alive.

It seems the request has not responded. How can I fix it? Any ideas?

In general please

post once
explain what you're doing & how to reproduce the problem
start a new issue for a new issue

In general please

post once
explain what you're doing & how to reproduce the problem
start a new issue for a new issue

Sure. Just deleted the duplication.


feat: bootstrapped basic web rendering using jinja2

Introduced Jinja and pydantic models so user input can be validated
Ability to create an organization
Dashboard view to list organizations
Rudimentary error handling (needs to be refactored to common error handler)

Before we land this, can we take it one step further to prove out i18n? We ran into this with the old site that integrating i18n caused us to throw out two templating systems where it was .... difficult to integrate. Would like to prove out the support before we decide on the library.


Provide helm chart to easily package / install operator

I've already started on one for a personal project of mine, but would like to port it upstream at some point so we don't need to maintain it in two places.
https://github.com/mjpitz/rabbitmq-operator-helm

This repo has been consolidated in to my charts repo so that I can host the helm chart repository via github pages: https://github.com/mjpitz/charts


CD and NCD profile report  Public Health Indicators changes

NCD Profile
To please replace Laboratory results recorded in the Public Health Indicators section with more specific indicators:
Patient have smoking status recorded
Diabetes mellitus patients have HbA1C recorded
Hypertension patients have BP recorded
To please replace Prescribing practice recorded  with a more specific indicator Availability of prescribed medicines calculated as would be medicine_available1 = yes / medicine_name1 (any entered)
CD Profile
To please remove Laboratory results recorded indicator for now
To please replace Prescribing practice recorded  with a more specific indicator Availability of prescribed medicines calculated as would be medicine_available1 = yes / medicine_name1 (any entered)

What french translation shall I use for "Availability of prescribed medicines" ?


Disabling check for conditional references

$Import - Disabling check for conditional references in incremental mode

/azp run


This is the website.

Unzipped is better :)

I'd put it just on the root e.g. under a folder 'website', putting it in the resources will include all files on the bot's .jar

The website should be in the resource otherwise the server cannot serve it. I think bundling both together make sens, WDYT?

I was thinking on using GitHub pages with a custom domain. It's a microsite, embedding it on the bot should be fine. LGTM


Stats Page

/stats page 📊
This PR introduces a new /stats page that can display all sorts of things!
Thoughts / Ideas
Here are some fun thoughts or ideas for the /stats page

Some estimation of current player count - This PR tries to introduce that by counter Twitch viewers for EFT to get a "pulse" on the game
Server uptime as reported by BSG
Average wipe length (we already have this but could display it here)
Most popular flea market item
Total website page views
Total website requests per second
Total items in game
Total registered EFT accounts (?)
Days since Nikita's last tweet

Fixed and got it working. Now to make it pretty

This is still a WIP


Compiling Slang to other shading languages (Metal, ESSL)

Is the Slang compiler able to generate Metal or ESSL shaders instead of GLSL?
If the Slang compiler doesn't already have this feature, then it could probably use SPIRV-Cross or ShaderConductor to generate Metal and ESSL shaders.

Support for more targets is always desirable, and we’d welcome pull requests that work toward support for the Metal shading language ESSL, etc.
The existing GLSL output from Slang can sometimes be used on OpenGL, but we currently don’t have logic to avoid/detect Vulkan-specific features. A back-end to support the OpenGL ES shading language (and/of WebGL) would probably need to start with making our GLSL output more compatible with desktop OpenGL.
Using another stage of translation on the output from Slang (as you describe) could be useful as a short-term solution for users who need support for platforms other than the current ones. The risk on that path is that the output for Metal, ESSL, etc. becomes limited to the “lowest common denominator” features of what is exposed in SPIR-V.
In the long run we believe that a dedicated Metal back-end will be worthwhile because then it will be possible for Slang to expose and take advantage of features that are only available on that API.
Our goal with Slang is that it allows for portable code when you want it, but also supports writing code that uses API-specific features when you need to.

Lets have a conversatiton about this in Q2 --- theres a priority thing but keeping this so we have the conversation at the right time

We're working on Metal support. Planning to support WGSL. Will track work on separate tasks.


Frontend - Browser refreshing on certain path is broken.

Describe the bug
During implementing frontend new navigation to user detail (new history) page (#1542), it appears there is an issue when refreshing the browser the page is completely broken.
The intended path (on browser address bar) for the new page/route is http://localhost:5173/user-details/users/9/applications/2. User refreshes the page, will triggers the application to issue request to backend with 404 error. It can be seen from the network tab that the application is getting a resource with wrong requesting path:

One other observation is this does not happen on other existing page when refreshing the browser.
Expected behavior

No fetching or loading error.
Page is refreshed and page is still showing the content correctly.

Screenshots
See above description.
Additional context

Some other existing page refreshing is still good, but this page appears to have issue.
This could be likely due to Vue Router configuration or Vite setup, not yet clear where it is causing the issue as some other page refreshing are still good.
At the time of this issue ticket was written, the new component/page is just a placeholder (under construction).

Shouldn't happen after refactoring work (tickets #1540).


chore: Pyarrow12 support

Feature or Bugfix

Bugfix

Detail

Pyarrow12 support

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: 8f948fd20bac5fb7dabdb47444cb5f96b913e87f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
Commit ID: 8f948fd20bac5fb7dabdb47444cb5f96b913e87f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: b9ca0603c062effaa8d11c0b9c7005b49a13a76f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: 0afeb1fba73a2a86308d102d6d677d85bb6d8090
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
Commit ID: 0afeb1fba73a2a86308d102d6d677d85bb6d8090
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: 2a5fa1755e790441eb7dfc573ddd3f0660f592e9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
Commit ID: 2a5fa1755e790441eb7dfc573ddd3f0660f592e9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: 403e818a6f78d0376cf5ab26201c52f2fcf68314
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Added Microsoft Edge WebView2 Runtime version 100.0.1185.44.

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

I accidentally pushed two manifests to the same branch :(


Input range not updating on attribute changes

When nanomnorph updates <input type="range"> they don't actually re-render untill they change size or you interact with them.

Morphdom somehow didn't have this issue.

This maybe?
https://github.com/patrick-steele-idem/morphdom/blob/master/src/specialElHandlers.js#L22-L40

No, thats not it.  Nanomorph also does this.


Update docs to show how to define scoped registries in 2019.4.12f1+

Once we move to 2019.4 LTS we'll likely lock onto a version older than 12f1, which introduced the ability to define scoped registries in the project settings in the Unity editor itself. We should add this to the getting started and downloading documentation.

I'll pick this up this weekend, now that we have moved to 2019.4


{Release} Upgrade to Azure CLI 2.38.2

Update version to 2.38.2
Resolve #28631

Update version to 2.38.2