In beta.21 I get an error about a missing entry points configured in a dependency

Pub Serve terminated
/usr/local/bin/dart/dart-sdk/bin/pub serve test --port=42943
Loading source assets...
Loading sass, angular2 and dart_to_js_script_rewriter transformers...
The value "web/index.dart" in "entry_points" of the Angular 2 transformer configuration does not point to any file.
Error on line 24, column 5 of ../../../contributing/ng_bootstrap/pubspec.yaml: Error loading transformer: Directory listing failed, path = './web/components/accordion/' (OS Error: No such file or directory, errno = 2)
    platform_directives:
    ^^^^^^^^^^^^^^^^^^^^^
Pub Serve terminated

pubspec.yaml
dependencies:
  angular2: ^2.0.0-beta.19
  bootstrap_sass: 4.0.0-alpha.2
  browser: ^0.10.0+2
  http: ^0.11.3+9
  meta: '^1.0.1'
  ng_bootstrap: #^0.1.0
    path: ../../../contributing/ng_bootstrap

transformers:
- sass
- angular2:
    mirror_mode: verbose
    platform_directives:
    - 'package:angular2/common.dart#COMMON_DIRECTIVES'
    platform_pipes:
    - 'package:angular2/common.dart#COMMON_PIPES'
    entry_points:
    - web/main.dart
    - test/webui/primitive_value_input/string_input_test.dart
    resolved_identifiers:
        BrowserClient: 'package:http/browser_client.dart'
...

I don't get why the Angular2 transformer reads entry_points from a dependency.

@natebosch Could you look and see if this is a bug from @jakemac53's commit?

If this is caused by the change I made, I guess an additional try - catch might do, so that missing entry points get printed but the transformer doesn't fail.

It would be better to only check entry_points from the current project. I assumed this to be the default.
@jakemac53 any hint about how to avoid this problem would be appreciated.

I don't believe this is related to 7e92b19
I tried reverting that commit and I can still repro.

Git bisect points to 8cb776f which makes sense - but doesn't help us pinpoint where the extra entrypoints are sneaking in. In any case I think they existed before that commit went in.

@zoechi Does this still need be P1 - rather, is this causing pain or was it a one-off issue?

@matanlurey
I'm using a local fork of ng_bootstrap where the entry_points (for the examples) are commented out
Yes, I'd consider this P1

OK, how can I reproduce - get the latest version of ng_bootstrap? @zoechi

Since https://github.com/dart-lang/angular2/pull/71 is going in does it need to be P1?
I believe this behavior of the transformers and pub has always existed. Using an angular app with entry_points defined as a dependency wasn't something that was anticipated.
It might make sense to either change the behavior of pub, or handle this better in the angular transformer. It looks like we're doing IO and we shouldn't be in a transformer so it's possible there is something we could fix without changing pub. In either case it's a pretty deep change.

@matanlurey Using any package as dependency that has the angular2 transformer with entry_points configured.
https://github.com/dart-league/ng_bootstrap/blob/develop/pubspec.yaml still has them, therefore adding it as dependency should do.


common log format error

burger.letters.com - - [01/Jul/1995:00:00:12 -0400] "GET /shuttle/countdown/video/livevideo.gif HTTP/1.0" 200 0
the ip of type burger.letters.com is not getting parsed correctly

Which Docker Container do you use?

In any case here is pattern you need:

- #  Web Logs
  sourceName: !!js/regexp /httpd|access_log|apache2|nginx/
  match:
    - regex: !!js/regexp /^(\S+)\s+(-|.+?)\s+(-|.+?)\s+\[(.*)\]\s\"(\S+)\s(\S+)\s(\S+)\s(\d+)\s(\d+)/i
      type: access_common
      fields:       [client_ip,remote_id,user,ts,method,path,http_version,status_code,size]
      dateFormat: DD/MMM/YYYY:HH:mm:ss ZZ
      transform: !!js/function >
        function (p) {
          p.message = p.method + ' ' + p.path
        }

The pattern is now in https://github.com/sematext/logagent-js/commit/9d9b5d225765e800e38ce61ceebd830946531be3 and will be in the next npm version 1.1.18 after some tests with NGINX, Apache HTTP and other web servers. But you can download https://raw.githubusercontent.com/sematext/logagent-js/master/patterns.yml and use this file via command line
wget https://raw.githubusercontent.com/sematext/logagent-js/master/patterns.yml -o mypattern.yml
tail -f /var/log/access_log | logagent  -f ./mypattern.yml -y -n httpd

Or use in sematext/sematext-agent-docker
docker run ... -v $PWD/mypattern.yml:/etc/logagent/patterns.yml


OffScreen Drawing

Not sure if this is related to #120 so here goes...
I've got a stack of identically sized FMX TSkPaintBox-es, each contain images. Then I've got a SKRuntimeEffect Shader that alters the images based on colour and/or transparency to provide paint-program style 'layers'. When this renders to the screen I end up with a layered image being displayed - all this works perfectly.
However, the images may be (much?) larger than the TSkPaintbox so the Shader also scales so the displayed composite looks correct.
What I want to do is to be able to render the composite at full resolution so that e.g. a displayed but scaled 4k image renders to the full 4k off-screen and can then be saved to  disk as output.
There are two methods that occur - the preferable solution is to render the result using Skia, this is desirable as the Shaders already do everything.
If this is not possible (or too much hassle) I can always replicate the Shaders in Pascal and build the resultant image that way.
If this is trivial a small FMX example would be very much appreciated (I'm sure it would also be of interest to other users of this project as the technique is really useful)

Hi @peardox!
Basically what you want would be to save a screenshot of your TSkPaintBox(es) with a custom size? There are several ways to do this, but if the controls are just TSkPaintBox and they all have the same dimension, it could be done very simply:
procedure SaveLayersScreenShot(const AWidth, AHeight: Integer);
var
  LSurface: ISkSurface;
begin
  LSurface := TSkSurface.MakeRaster(AWidth, AHeight);
  LSurface.Canvas.Clear(TAlphaColors.Null);

  // SkPaintBox1 - Layer 1
  if Assigned(SkPaintBox1.OnDraw) then
    SkPaintBox1.OnDraw(SkPaintBox1, LSurface.Canvas, RectF(0, 0, AWidth, AHeight), 1);

  // SkPaintBox2 - Layer 2
  if Assigned(SkPaintBox2.OnDraw) then
    SkPaintBox2.OnDraw(SkPaintBox2, LSurface.Canvas, RectF(0, 0, AWidth, AHeight), 1);

  // Layer n...

  LSurface.MakeImageSnapshot.EncodeToFile('big-screen-shot.png');
end;

Thanks a million @viniciusfbb - your helpful attitude is really appreciated, far less people than you'd imagine have your kind of self-motivated helpfulness.
While this may be simple to some it's not to others so it really does deserve adding to the README.md
I'll go try it out immediately, thanks again.

Hmm - it's extremely slow. I'm timing c. 12 seconds for a test 2 layer 2400x3000 image (decent laptop)
Still, it does work as desired.
On screen it's super-fast of course

Oh yes. What is happening is that the canvas of the screen is rendered on the GPU, while the canvas of surfaces created by MakeRaster (our code) are rendered on the CPU. There is generally no performance problem drawing on raster (CPU) surfaces, except for heavier operations like shader. In this case, let's try another code that creates a surface that is rendered internally by the GPU:
uses
  Skia.FMX.Graphics;

procedure TForm1.SaveLayersScreenShot(const AWidth, AHeight: Integer);
var
  LSurface: ISkSurface;
begin
  if (Self.Canvas is TGrCanvasCustom) and Assigned(TGrCanvasCustom(Self.Canvas).Context) then
    LSurface := TSkSurface.MakeRenderTarget(TGrCanvasCustom(Self.Canvas).Context, False, TSkImageInfo.Create(AWidth, AHeight))
  else
    LSurface := TSkSurface.MakeRaster(AWidth, AHeight);
  LSurface.Canvas.Clear(TAlphaColors.Null);

  // SkPaintBox1 - Layer 1
  if Assigned(SkPaintBox1.OnDraw) then
    SkPaintBox1.OnDraw(SkPaintBox1, LSurface.Canvas, RectF(0, 0, AWidth, AHeight), 1);

  // SkPaintBox2 - Layer 2
  if Assigned(SkPaintBox2.OnDraw) then
    SkPaintBox2.OnDraw(SkPaintBox2, LSurface.Canvas, RectF(0, 0, AWidth, AHeight), 1);

  // Layer n...

  LSurface.MakeImageSnapshot.EncodeToFile('big-screen-shot.png');
end;

You're my hero!

Well, that's much better - 1.5 secs but...
I'm not sure if this is a bug or not....
After it saves the image the form no longer updates or responds to clicks.
At first I thought that I has something wrong in the label that displays the timing so I added a ShowMessage with the timing just to check (which is how I know how long it takes)
It appears that OS controls still function but other stuff don't. Resizing the form results in a blank form...
If you need a quick test just grab https://github.com/peardox/Compositing compile + run then select Save Composite from File menu then everything will stop responding. Sorry I ain't got anything simple to hand...

Excellent! About the problem, I have to do some tests, maybe it's the lack of FlushAndSubmit, or the lack of a MakeRasterImage at the end... The truth is that it's quite annoying to mess with GPU surfaces, especially when they are global variables (because they would need to register some messages to detect context destruction), but in your case it's just a local temporary surface, so it won't be complicated.
I'm just going to do some tests tomorrow to adjust this code I gave you.


Handle Loss of Deprecated Container Fields in v1 for Step LivenessProbes and etc.

Feature request

In pipeline/pkg/apis/pipeline/v1beta1/container_conversion.go v1beta1 we are converting a v1beta1 sidecar to a v1beta1 step, which doesn't have any information loss while during the v1-storage swap, converting a v1 sidecar to a v1 step would have information loss.
The script.go needs to be updated to ensure these fields are still supported in sidecars, and add tests for that functionality if they don't already exist.
Related to https://github.com/tektoncd/pipeline/issues/4546
Use case

hi @JeromeJu
i would like to work on this.

/reopen

Pipelines WG - in progress, expected to close this in 0.48


test UDPv6Tests fails on OS X

http://ci.ros2.org/job/ci_osx/1210/testReport/junit/projectroot.test.unittest/transport/UDPv6Tests/

@dirk-thomas I made same changes that I did to resolve UDPv4Tests on OSX. Can you check them in your CI? Whether it still fails, reopen this issue. Thanks.

The test still fails in the latest CI build: http://ci.ros2.org/job/ci_osx/1234/testReport/projectroot.test.unittest/transport/
I don't have the permission to reopen this ticket. So please reopen it.

Maybe it's not the same issue as UDPv4Test on OSX. I will check it thoroughly.

The test also has a compiler warning: see http://ci.ros2.org/view/nightly/job/nightly_osx_debug/143/warnings8Result/

Any update on this? It still had two compiler warnings and fails a test: http://ci.ros2.org/job/nightly_osx_debug/160/

@richiprosima Ping

According to this link there is a problem using boost::asio::set_option() and multicast on OSX. Meantime I've disabled the test on OSX.
I've pushed the changes on branch hotfix/55 to be tests in your CI server.
If the changes work successfully, I will merge it in master branch.

I ran another build on OS X which doesn't have the test (failure) anymore: http://ci.ros2.org/job/ci_osx/1276/
So the branch can be merged as far as I can tell. Thanks.

@richiprosima Could you please also look into the remaining warnings: http://ci.ros2.org/job/ci_osx/1276/warnings9Result/ Those are the only ones keeping us from having a green build (at least sometimes :wink:).

I've updated branch hotfix/55, removing the warnings. As quick as you are agree with the changes, I will merge the branch in master.

The new OS X build (http://ci.ros2.org/job/ci_osx/1277/) looks good. +1 to merge this. Thank you.

Merged in master


Time elapsed displays incorrectly

As the output length changes, parts of the elapsed time don't get erased and appear at the end of the output.

This issue is tied into the plan to make the status output more accurate to FreeBSD's PKG.


Allow access to the operation id

I'm doing some Serilog integration currently in ASP.NET Core 3, where it would be really nice to be able to access the current operation_id and operation_ParentId, to ship that off along with the traces sent to Application Insights. Particularly in a situation where we use telemetryClient.StartOperation<RequestTelemetry>()
An earlier issue , suggested System.Diagnostics.Activity.Current?.RootId - but that's not accurate anymore. Looking through the codebase it seems like the
operation_Id is equivalent to Activity.TraceId and the operation_ParentId is constructed via $"|{activity.TraceId}.{activity.Parent.SpanId}." inside W3CUtilities.FormatTelemetryId
Is this:
a) Correct?
b) Guaranteed to be stable, if I end up creating this logic into my logging integration
c) Or it is perhaps possible to expose the functionality for getting these ID's, so we don't have to go through all of these hoops

OperationId in application insights will be activity.traceid.
ParentId is $"|{activity.TraceId}.{activity.Parent.SpanId}" to maintain backward compatibility with old SDKs. I believe parentid will be replaced with just activity.parent.spanid in the future, when request.id,dependency.id will be made activity.current.spanid instead of the current "|traceid.spanid"
If you use the SDK with default settings, then all traces (including ilogger ones) should get automatically tagged with correct parentid. Is this not what you see already? Perhaps you are hitting this issue (https://github.com/microsoft/ApplicationInsights-dotnet-logging/issues/305) ?
@lmolkova is the expert here to help more.

@GeeWee you should always make sure you configure OperationCorrelationTelemetryInititalizer and never stamp operation_Id, parentId or id on your own.
OperationCorrelationTelemetryInitializer will make sure telemtery is initialized properly.

That makes sense. The reason I'm asking about this, is because the Serilog AI Sink doesn't seem to properly instrument the log calls with operation_id or operation_parentid, and I was hoping I could simply monkey-patch it in with a custom logging enricher.
However, what you're saying is that there's potentially something more fundamental wrong in the sink, that should be fixed instead of monkeypatched?

Looking into the sink, it seems it uses TelemetryConfiguration.Active which always has OperationCorrelationTelemetryInitializer. However, I might have missed something, or maybe Serilog sink overrides ids somehow. Can you post here example of ids you see with serilog?

Yes, let's try to explore the sink issue. Perhaps this is more of an issue over there - if we manage to resolve this, I'll make a PR over there.
While the documentation says to use Telemetry.Active, there's also a different constructor that people have been playing around with in #121.
We're not sure TelemetryConfiguration.Active is safe to use, seeing as the configuration made in ASP.Net Core's Configure might not be applied.
Digging a little deeper in the different constructors, I find I've been using the following one in the sink
// Inject IOptionsMonitor<TelemetryConfiguration> inside Startup.Configure
            var loggerConfiguration = LoggerConfiguration
                // other stuff
                .WriteTo.ApplicationInsights(telemetryConfiguration, TelemetryConverter.Traces);

This leads to the following code.
var client = new TelemetryClient(telemetryConfiguration)
Which is then used internally in the sink.
Is this an inaccurate way of constructing the TelemetryClient?
As alternatives, I've just now tried the sink constructor that uses the ASP.Net Core TelemetryClient and uses that, but also the one that uses TelemetryConfiguration.Active - both send what I assume are the correct operational_id's, though I haven't double-checked. When using the constructor above, no operation_ids of any kind are sent.

We're not sure TelemetryConfiguration.Active is safe to use, seeing as the configuration made in ASP.Net Core's Configure might not be applied.

Do not use Active if you can avoid it. AI SDK attempts to initialize it in the same way as one in DI, but any customization may lead to discrepancies. With logger adapters it's not always possible NOT to use it though.

var client = new TelemetryClient(telemetryConfiguration)

This is the proper way to initialize TelemteryClient. What I'm mostly interested in is where telemetryConfiguration instance came from and how it was initialized. My guess is that that configuration does not have OperationCorrelationTelemetryInitializer - you could always check the list of initializers on the configuration. Do you own this instance or it's created by the sink?
If you own it - create it with TelemteryConfiguration.CreateDefault() rather than parameterless constructor. If you don't - this is worth fixing in the sink.
As a workaround you could always add initializer to any instance of TelemteryConfiguration.

As alternatives, I've just now tried the sink constructor that uses the ASP.Net Core TelemetryClient and uses that, but also the one that uses TelemetryConfiguration.Active - both send what I assume are the correct operational_id's, though I haven't double-checked.

This is also a reasonable workaround.
Let me know if it anwsers your question. Thanks!

Alright, so I think I've tried to dig up the problem.
The instance owns the sink, but there's a mismatch in the TelemetryInitializers between the ASP.Net core injected TelemetryClient and the TelemetryConfiguration.
If In Configure I inject both TelemetryClient and IOptionsMonitor<TelemetryConfiguration>, I see the following TelemetryInitializers on the the TelemetryConfiguration
[0] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.ClientIpHeaderTelemetryInitializer}
[1] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.OperationNameTelemetryInitializer}
[2] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.SyntheticTelemetryInitializer}
[3] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.WebSessionTelemetryInitializer}
[4] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.WebUserTelemetryInitializer}
[5] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.AspNetCoreEnvironmentTelemetryInitializer}
[6] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.DomainNameRoleInstanceTelemetryInitializer}
[7] = {Microsoft.ApplicationInsights.WindowsServer.AzureWebAppRoleEnvironmentTelemetryInitializer}
[8] = {Microsoft.ApplicationInsights.DependencyCollector.HttpDependenciesParsingTelemetryInitializer}
[9] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.ComponentVersionTelemetryInitializer}

And the following on the TelemetryClient
[0] = {Microsoft.ApplicationInsights.Extensibility.OperationCorrelationTelemetryInitializer}
[1] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.ClientIpHeaderTelemetryInitializer}
[2] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.OperationNameTelemetryInitializer}
[3] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.SyntheticTelemetryInitializer}
[4] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.WebSessionTelemetryInitializer}
[5] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.WebUserTelemetryInitializer}
[6] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.AspNetCoreEnvironmentTelemetryInitializer}
[7] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.DomainNameRoleInstanceTelemetryInitializer}
[8] = {Microsoft.ApplicationInsights.WindowsServer.AzureWebAppRoleEnvironmentTelemetryInitializer}
[9] = {Microsoft.ApplicationInsights.DependencyCollector.HttpDependenciesParsingTelemetryInitializer}
[10] = {Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.ComponentVersionTelemetryInitializer}

So the Configuration is missing the OperationCorrelationTelemetryInitializer, that is attached to the client.


Schema Builder Parameters: let users specify the units

See: https://speckle.community/t/revit-unit-conversion/1948
Potential steps to take:

[x] allow users to specify their own units in ParameterUpdater
[ ] add more unit types (and conversions) to Core


⚠ Adacta-fintech.com has degraded performance

In 5163848, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 502
Response time: 526 ms

Resolved: Adacta-fintech.com performance has improved in 905b4b5 after 7 minutes.


when using .mbconfig with git, train time gets reset

I trained a regression model locally on my machine for 60 seconds and then saved and pushed changes to git
@luisquintanilla pulled the changes and opened the mbconfig - in the Train screen, the state of the last training run was saved and all there, but the train time in the textbox was set to 600 - we opened the .mbconfig json and saw the train time was still set to 60
Luis added a Web API from Consume step and saved everything
Luis used the ML.NET CLI to train the same .mbconfig and the train time was 600 seconds in the CLI - we opened the .mbconfig json again, and now the train time said 600 seconds

I suspect that we're use the calculated value instead of the value from the training config. We should use the training config value. When the training time value is 0 (like on first initialize) then we should run the calculation.

LGTM


setting email configuration - self-signing certificate error

**Hi guys,
I'm new in 4minitz. I have configured the email as below:**
_"//3": "below option configures if and how to send E-Mails",
"email": {
"//3.1": "Switch on/off E-Mail sending",
"enableMailDelivery": true,
"//3.2": "Choose one of the below mail deliver methods",
"mailDeliverer": "smtp",
    "//3.3": [
        "defaultEMailSenderAddress: if this is an empty string, the sending users mail address will",
        "be the 'FROM:' sender of the email. If you specify an email address here, this mail address",
        "will be the 'FROM:' sender of each mails, and the sending user will be inserted to the 'REPLY-TO:' field",
        "If you set this to empty, please specify fallbackEMailSenderAddress!",
        "(see admin guide for the decision tree)."
    ],
    "defaultEMailSenderAddress"<EMAIL_ADDRESS>
    "//3.4": [
        "defaultEMailSenderExceptionDomains: an array of domains.",
        "For these domains the defaultEMailSenderAddress will NOT be used.",
        "Use case: Inside your intranet you want to send mails from real user names of you company.",
        "But if you cooperate with users from other companies inside your 4Minitz, these",
        "users will get trouble sending mail from your 4Minitz to mail addresses",
        "inside their company with their mail address in FROM: header.",
        "This really looks like spam, so it will probably be blocked!."
    ],
    "defaultEMailSenderExceptionDomains": ["faa.net.my", "faa.org.my"],

   "//3.5": [
            "fallbackEMailSenderAddress: if sending user has no mail address specified in the user profile,",
            "this will be 'FROM:' sender.",
            "If defaultEMailSenderAddress is set empty this field must be set with a valid email address.",
            "If this setting is empty and user has no mail address in profile, sending eMail will fail!",
            "(see admin guide for the decision tree)."
        ],
    "fallbackEMailSenderAddress"<EMAIL_ADDRESS>    "ignoreSslErrors": true,
    "webSecurity": false,
    "strictSSL": false,
    "allowSelfSignedTLS": true,

    "smtp": {
        "mailUrl": "smtp://username:password@mail.faa.net.my:587/"
    },_

When testing sending Agenda by clicking the SEND AGENDA and check the log, below is the error log.
*** ROOT_URL: http://1**.1*.0.1**:3100
WebApp current working directory:/4minitz_bin/bundle/programs/server
#Email could not be sent successfully to<EMAIL_ADDRESS>Email subject: [CSD - Admin Team] Audit Issues (IT & Admin): Procurement on 2018-05-03 (Agenda)
Error: Error: self signed certificate
#Email could not be sent successfully to<EMAIL_ADDRESS>Email subject: [CSD - Admin Team] Audit Issues (IT & Admin): Procurement on 2018-05-03 (Agenda)
Error: Error: self signed certificate
#Email could not be sent successfully to<EMAIL_ADDRESS>Email subject: [CSD - Admin Team] Audit Issues (IT & Admin): Procurement on 2018-05-03 (Agenda)
Error: Error: self signed certificate

Is this issue still open on your side?
I'm not an SMTP/TLS expert. And honestly, we most of the time use unencrypted, local SMPT servers. So one shot from my side: If you start 4Minitz via docker, then try the following:
add a -e "NODE_TLS_REJECT_UNAUTHORIZED=0" after your docker run command.
Please report back if this helps...

No feedback.
Nevertheless thanks for reporting!
But we will close this issue for now.
Please reopen, if issue persisist.


Remove dev dependency on the runtime crate

Resolves #333 by removing the development dependency on runtime.

a separate PR to add the http-service-hyper example would be great.

I'll PR a new example shortly.


How to autofocus on file that is currently active ?

I want to highlight my current opened file in the tree structure. How ?? please

See the readme
require("neo-tree").setup({
    filesystem = {
         follow_current_file = true
   }
})

It looks like this isn't in the help doc, though it is on the main readme of the project.

Whoops, accidentally closed this lol, we need to update the help doc to include this option

@miversen33 The help doc specifies that not all configs are documented there but in default.lua instead (which makes sense, as there are lots of configs), so I don't think we'll need to add it there.
I'll close this issue as solved, but feel free to reopen it if you think we should add it to the help doc.


SSL mode should be require not required

This has been fixed already recently in the 0.8 branch (see DBZ-711, there were some other wrong values, too). It will be published to the website very soon, so I'm going to close this one. Thanks nevertheless, @l0n3r4n83r!


Multi AppSource Apps in one repo.

Im trying to publish multiple apps in one repo to appsource.
You can only enter one productId per .AL-Go/settings.json. So I tried to make the repo multi project but now I get the following error.
Error: RunPipeline action failed. Error: For AppSource Apps with AppSourceCop enabled, you need to specify AppSourceCopMandatoryAffixes in .AL-Go\settings.json Stacktrace: at AnalyzeRepo, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\AL-Go-Helper.ps1: line 539 at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\RunPipeline\RunPipeline.ps1: line 79 at <ScriptBlock>, D:\a\_temp\d965a261-a9b2-4f1c-91d7-c20ad8eabeb2.ps1: line 1 at <ScriptBlock>, <No file>: line 1
https://github.com/microsoft/AL-Go-Actions/blob/53470cc1bf0524559a158d90b093170d7fb3ec76/AL-Go-Helper.ps1#L539
It seems like the analyse-repo part doesn't respect multi project AppSource Repos.
I'm doing something wrong here?

I found a solution

.AL-Go/settings.json - AppSourceCopMandatoryAffixes set to any value
app1/.AL-Go/settings.json - AppSourceCopMandatoryAffixes set for the app1
app2/.AL-Go/settings.json - AppSourceCopMandatoryAffixes set for the app2

But this feels like cheating and this solution drasticly increases the build time.
Whats the reasoning to only allow one appsource app per project?

One project is one shippable entity, you can have multiple apps in one project, but they would be under one appsource app.
Seems like a bug that you cannot have a multi-project repo with 2 appsource app projects - the above "solution" makes the root into a third project - and is likely not what you want.

If you set appFolders to an empty error no harm
Is there anyway to eliminate wasted build time here? Maybe we can cache container images if all projects have the same requirements?
Because if you capsulate your code you will often create bridge apps, which are independently deployable, but are still tightly coupled.

There is no "good" way to get dependencies between projects within the same run.
The following option is problematic
  "appDependencyProbingPaths":  [
        {
            "repo":  "https://github.com/<org>/<repo>",
            "release_status":  "latestBuild",
            "projects":  "*"
        }
    ]

because this depends on the build to be successful before it can be used. So this will use the artifacts from the prevoius run and not the latest.
After release this might be okay. But during development you will often make changes in both projects.
Specially with bridge apps, they are both shippable units but still tightly coupled.
So the only solution I see here is to create a repo for each app but this makes working with lots of apps very difficult. For us this would currently mean 30 additional repos.

There are two solutions to this.

Set "UseProjectDependencies" to true in .github/AL-Go-Settings.json and run update AL-Go System files.
Use release_status = include in your probing paths.

The first one will turn your build pipelines into multi-stage pipelines.
The second one will build the dependent apps together with the depending app.
The first one causes build time to shoot up - but over time (with up-coming performance improvements) this should go away again.
My thinking is that I can build apps in all stages and run all tests in the latest stage, to avoid creating containers for all stages.

Thanks, I'll give it a try.
I think both of these setting are missing in the documentation right?
I looked at the following part.
https://github.com/microsoft/AL-Go/blob/main/Scenarios/settings.md

will add scenarios for these

The build worked correctly, but I now get this error when delivering to nuget.
D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver
Downloading BcContainerHelper latest version from CDN
BcContainerHelper version 4.0.7
BcContainerHelper emits usage statistics telemetry to Microsoft
Reading .github\AL-Go-Settings.json
Reading .AL-Go\settings.json
Artifacts D:\a\Project\Project\.artifacts
Projects:
Project1
Project2
.
Using GitHubPackagesContext
Project 'Project1'

    Directory: D:\a\Project\Project\.artifacts

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----          1/2/2023   5:58 PM                Project1-main-Apps-<IP_ADDRESS>
d-----          1/2/2023   5:58 PM                Project2-main-Apps-<IP_ADDRESS>

Error: Deliver action failed. Error: Internal error - multiple Apps folders located Stacktrace: at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver\Deliver.ps1: line 197 at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver\Deliver.ps1: line [193](https://github.com/byndit/BeyondAttributes/actions/runs/3824202039/jobs/6506328816#step:7:195) at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver\Deliver.ps1: line 72 at <ScriptBlock>, D:\a\_temp\355272af-02c0-4b9e-8f05-ab384f03287e.ps1: line 1 at <ScriptBlock>, <No file>: line 1

I'm not sure if this is still a bug in the nuget delivery or if this is caused by having the third project in base.

I will check this.

Looking at the first error first - created a multi-project repo with 2 appsource apps - I cannot repro the first issue.
When AnalyzeRepo is called - settings is repoSettings AND project setting for the project being built.
I can repro the issue with deliver to NuGet failing and will look at that.

There is a fix for this issue on https://github.com/freddydk/AL-Go-AppSource@issue328 - if you want to try it.
I will run this through end 2 end testing and some more tests.

Thanks, the nuget issue is fixed.
I've created a repo that reproduces the "error".
But I know understand why this is happening. If you use multi project repos you need to run "Updated System files" if you add or remove projects. But this can't be done using a pr. Because the workflow will run on the old workflow which might have had to change.
https://github.com/byndit/AL-Go-Issue138/pull/2/checks?check_run_id=10518386337
But this is probably just something you need to know. Because I'm not sure whats the solution for this.

Correct - if you use UseProjectDependencies - you need to run Update AL-Go System Files if you change the dependency depth (with the workflow will tell you).
Just adding/removing projects should not require you to run the update workflow.
The error you have now is actually a new error in my issue328 branch - any chance you can give me write access to your repo - then I will add a fix directly

Wrong - the error is not new, the consequence is new.
You should be able to re-run the jobs in the PR workflow now

It works now. Feel free to close the issue once this is merged into al-go. I'll then delete the repo again.


No PowerShell support Run Command for Linux

Unlike the CLI, it seems that the PowerShell Invoke-AzVMRunCommand cannot be used to run scripts inside Linux VMs. The script must be present on the calling machine, which is not practical.
The problem is that Azure Automation runbooks only support PowerShell and not CLI. Is there any solution to this catch-22?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 792f4871-0d07-624f-7daf-02de97121e13
Version Independent ID: b70d481f-b351-bb38-cb49-1cda63422968
Content: Run shell scripts in an Linux VM on Azure
Content Source: articles/virtual-machines/linux/run-command.md
Service: automation
GitHub Login: @georgewallace
Microsoft Alias: gwallace

@dohughes-msft Thanks for the Comment. We are actively investigating and will get back to you soon.

@dohughes-msft Run command does not require an Automation account to work.  Is there somewhere you are seeing states this? As for running the scripts, az cli is only used for Linux machines and PowerShell is only used for Windows machines. Can you please clarify what it is you are having issues with?

@georgewallace yes that's correct, I want to execute a script inside a Linux VM from an automation runbook. CLI is not available for that.

@dohughes-msft So that is a problem, there are two ways to solve that. Run it from a hybrid runbook worker where you can install whatever you need locally. Or use the REST API endpoint for run command... You can call it with PowerShell with the example below replacing the values in the uri with your valid values. I just used an example of running ps aux.. you can do whatever you want though.
$azContext = Get-AzContext
$azProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
$profileClient = New-Object -TypeName Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient -ArgumentList ($azProfile)
$token = $profileClient.AcquireAccessToken($azContext.Subscription.TenantId)
$authHeader = @{
    'Content-Type'='application/json'
    'Authorization'='Bearer ' + $token.AccessToken
}

# Invoke the REST API
$restUri = 'https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}/runCommand?api-version=2017-03-30'
$body = @{
    'commandId' = 'RunShellScript'
  'script' = @('ps aux')
}
$response = Invoke-webrequest -Uri $restUri -Method Post -Headers $authHeader -Body $($body | ConvertTo-Json)

$asyncstatus = $($response.headers.'Azure-AsyncOperation')
$status = "InProgress"
While($status -eq "InProgress")`
{
Write-Output "Status: $status"
$response = invoke-webrequest -uri $asyncstatus -Headers $authHeader
$status = $($response.Content | ConvertFrom-Json).status -eq "InProgress"
sleep 5
}
Write-Output $($response.Content | ConvertFrom-Json).properties.output.message

@dohughes-msft  We will now close this issue. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.

save your block-script to file and reference the file. tested on automation account sandbox.

The script like "mkdir test" works for me where it creates folder inside the VM. However, iam not able to invoke a .sh file that is present in the VM. Tried doing  the below, but no luck. Any help is appreciated.
$body = @{      'commandId' = 'RunShellScript'      'script' = @('sh /home/azureuser/test.sh') }


Carbon statement with impact 0

Hello, very good project. Thanks for sharing. I have run my_experiment.py on a Linux virtual machine (Ubuntu) and then running the script generate_carbon_impact_statement with "USA" I get the following output: "This work contributed 0.000 kg of....", that is, with 0s in all indicators. Is this normal? Thanks in advance....

hi @pnovoa! Could you provide a minimal repro so it'd be easier for us to investigate? for example
your logs and  the command you used to generate the statement?
thanks!

hi @pnovoa! Could you provide a minimal repro so it'd be easier for us to investigate? for example
your logs and  the command you used to generate the statement?
thanks!

hi @pnovoa! Could you provide a minimal repro so it'd be easier for us to investigate? for example
your logs and the command you used to generate the statement?
thanks!

I have run the following command:
generate-carbon-impact-statement /tmp/tmp4b0gac2l/ "USA"
while the content of the folder tmp4b0gac2l is as in the zip file below:
tmp4b0gac2l.zip
Thanks

hi @pnovoa! Could you provide a minimal repro so it'd be easier for us to investigate? for example
your logs and the command you used to generate the statement?
thanks!

I have run the following command:
generate-carbon-impact-statement /tmp/tmp4b0gac2l/ "USA"
while the content of the folder tmp4b0gac2l is as in the zip file below:
tmp4b0gac2l.zip
Thanks

Hi,
So, it looks like the tool isn't able to find any power readings from either a GPU or CPU. Does the Ubuntu machine have nvidia-smi installed? Also do you see anything under the directory /sys/class/powercap/intel-rapl/ ?
For the tool to track energy, it needs to have certain firmware dependencies (Intel's RAPL and/or nvidia-smi installed). We've added an issue to prevent silent failures like this one and will update this issue once that's done. In the meantime, I think you need to set up your machine to expose Intel rapl and nvidia-smi to actually get any energy readings.

Hi,
So, it looks like the tool isn't able to find any power readings from either a GPU or CPU. Does the Ubuntu machine have nvidia-smi installed? Also do you see anything under the directory /sys/class/powercap/intel-rapl/ ?
For the tool to track energy, it needs to have certain firmware dependencies (Intel's RAPL and/or nvidia-smi installed). We've added an issue to prevent silent failures like this one and will update this issue once that's done. In the meantime, I think you need to set up your machine to expose Intel rapl and nvidia-smi to actually get any energy readings.

Hi, I assume this is resolved. If it's not, please let us know and re-open the issue. Thanks!

Hi, I assume this is resolved. If it's not, please let us know and re-open the issue. Thanks!


Allows spaces between column names for --columns and --names

It took me a little while to work out what was going wrong and this may be a little hard to fix.
Essentially I habitually add a space after a comma.
The result for me was that when specifying columns to the --column switch I was getting an error 'too many files'.
I eventually worked out that the --column switch looks for the first space to determine the end of the column list.
This isn't documented and the resulting error was non-obvious.
This either needs to be explicitly documented or the command line changed to deal with spaces.
It looks like change the command line parsing could only be done in a 'non backward compatible' method so it might be better to just highlight this fact in the doco.

I should note that I think the below should work:
gocsv select --columns "Description, Name, Remote, Label, Product, Description" x.csv
panic: Could not find header " Name"
Note the space between the comma and the Name column.

Good call. I just added some references to that issue in the documentation: https://github.com/DataFoxCo/gocsv#specifying-columns. Let me know if that doesn't seem sufficient.


Make the skeleton compile with noImplicitAny set to true

@valerian is currently working on this at the moment, just putting this here for issue tracking purposes.

Turns out this project has already been compileable with noImplicitAny set to true. We've made some typing fixes on #5 and wrote on the readme about how to switch the noImplicitAny flag to true.


request: add shiba metadata

I've created a pull request to add shiba on INJ metadata.
Thanks.

Will merge from a local branch.


Improve and guard revision-as-constraint handling

#51 left a nasty hole in dealing with #49: if a constraint specifies a revision that doesn't actually exist, it tosses out an error much later in the solve process than it maybe really should. This issue also extends to both preferred versions (#16) and lock versions.
There's a fair bit of logic included that's dedicated to ensuring specified revisions actually exist. But it's currently disabled, first because it's guarding against what seems like a really remote case (at least, with respect to the level of quality we need for MVP), and second because having it in there further, and kinda unconditionally, drags down performance.
There are some approaches that we could take to making this better, but they've all got tradeoffs:

Revision verification in checker step (the current, commented-out approach):

Pros:

Mostly central - not a lot of checks sprinkled all over
Clear (?) fail conditions presented to trace logger
Covers lockv and prefv cases as well

Cons:

Doesn't cover revision constraints declared by root (so, not fully central)
Deviates from existing checker pattern (in the way it busts on root)
VERY unfriendly to locked versions allowing a fast solve run

Revision lookups done as part of *versionQueue.advance()'s "all loaded" logic

Pros:

Again, mostly central - covers the above root case
Follows the fastpath for locked versions logic

Cons:

Does nothing for lockv or prefv verification, so not fully central

Optionally verify revisions at the end of solve

Pros:

Will probably jive nicely with prefetching, and thus the fastpath
Fully central

Cons:

Totally out of character with other checks
Weird failure location for nonexistent revs - at the end?
Potentially VERY inefficient - if a lockv rev failed in this way, this would pretty much invalidate the entire solve

We do have an implementation in now that follows path 1. I'm leaving this open, though, as it's something we might want to reconsider in the future.

This issue was moved to golang/dep#442


How to tune hyperparameters

Thanks for your great work! This repo includes many state-of-the-art methods and is easy to reproduce. So I would like to ask for some advice about insight into tuning hyperparameters. There are many hyperparameters given in the script of training, and I'm wondering how did you dig them out. How should I adjust them if I am training a model of my own?

@rwightman I'm also stumbled upon same question. How could I know what are the hyperparameters that I'm able to tune? I agree it varies depending on the model, but I'm looking for some documentation or a place to dig into. Particularly, I'm interested in customizing activation_layer, attention_layer, anit-aliasing layer

@liqi17thu @Kshitij09 this reply has been a long time coming, but going to move to discussions as it seems a good thread to leave open.
There's no magic to hparam tuning, just persistence and spending enough time with your models and datasets to get a feel for what works and what doesn't. With more experience you'll get better at finding starting points but will still have to search for the optimal values. If you've got enough compute, just doing exhaustive sweeps can be a good strategy, especially if your dataset isn't large.
The starting points, and large improvement jumps in hparams often comes from ideas put forward in various papers and their combinations. My current 'best' ImageNet hparams are based on a blend of ideas in EfficientNet, RandAugment, and 'Bag-of-Tricks' papers + experience based additions which include turning up the augmentation and regularization significantly. This won't necessarily work as well on a different dataset, especially smaller ones, or for fine-tuning instead of training from scratch, etc.
Fine-tuning, I often find a simple combo of Adam + Plateau LR schedule, or Momentum/Nesterov (SGD) w/ cosine decay and lower LR than you'd use for training from scratch is a good starting point. Also less augmentation is usually a good starting point.


Get logits/predictions from a detector instead of loss

Is there a way to load a detector, feed it an img and img_meta, and retrieve the class logits/predictions instead of the loss? It looks like the default behavior of forward_train and forward_test is to return the loss. After looking through the code, it looks like forward_dummy, https://github.com/open-mmlab/mmdetection/blob/547eb8d12c1b29ad9644160ca5dcb223cc1d0ce6/mmdet/models/roi_heads/standard_roi_head.py,  might give me what I need. Is this correct or is there a different way to achieve my goal? Thanks in advance!

Upon a deeper dive of forward_dummy, it look like it's generating random proposals; so, the bbox logits/predictions wouldn't be usable. Is there an easy way to return this information?

@hellock  and @xvjiarui,
The following code gets me the logits, as well as the bboxes. However, even though the model I'm using only has 12 classes, which I verified with print(model.CLASSES) and print(len(model.CLASSES)), I get a 1000x320 tensor for bboxes and a 1000x81 tensor for the logits. It looks like it defaults to the number of COCO classes. How can I override this to be the number of classes in model.CLASSES? Also, for the logits, is the first element of each row, the background class?
# build the model from a config file and a checkpoint file
model = init_detector(config=config, checkpoint=checkpoint, device=device)

img_metas = {}
channels, height, width = img.shape[-3:]
img_shape = (int(height), int(width), int(channels))
img_metas["img_shape"] = img_shape
img_metas["scale_factor"] = 1.0

img = torch.from_numpy(img).to(device).float()

with torch.no_grad():
    x = model.extract_feat(img)
    proposal_list = model.rpn_head.simple_test_rpn(x, img_metas=[img_metas])
    rois = bbox2roi(proposal_list)
    bbox_results = model.roi_head._bbox_forward(x, rois)
    logits = bbox_results["cls_score"]
    bbox_pred = bbox_results["bbox_pred"]
    bboxes = model.roi_head.bbox_head.bbox_coder.decode(rois[:, 1:], bbox_pred, max_shape=img_shape[:2])

Hi @CMobley7
You may check the num_classes of heads.

@xvjiarui, you're right. It looks like num_classes is hardcoded in most configs for the roi_head, mask_head, semantic_head and mask_iou_head. I see now that https://mmdetection.readthedocs.io/en/latest/tutorials/finetune.html, instructs one to change this variable, my bad. I was under the false impression that this would be set dynamically based off of one's dataset, i.e. the classes var in the config ( https://mmdetection.readthedocs.io/en/latest/tutorials/customize_dataset.html). Is there anyway, num_classes can be added dynamically via doing a len of the classes var?
Also, it appears that when I use tools/test.py, it outputs the right number of classes when doing evaluation and inference. However, when I examine the code, it looks like it should be outputting 80 classes based off of num_classes in the particular head, not the number of classes in my custom dataset. How is this possible?

Hi, I wrote a function that can return the class probabilities for each detection.
def inference_detector_with_probs(model, img):
    """Inference image with the detector.
    Args:
        model (nn.Module): The loaded detector.
        img (str): image file.
    Returns:
    """

    cfg = model.cfg
    device = next(model.parameters()).device  # model device
    cfg.data.test.pipeline = replace_ImageToTensor(cfg.data.test.pipeline)
    test_pipeline = Compose(cfg.data.test.pipeline)

    datas = []
    # add information into dict
    data = dict(img_info=dict(filename=img), img_prefix=None)
    # build the data pipeline
    datas.append(test_pipeline(data))

    data = collate(datas, samples_per_gpu=len(imgs))
    # just get the actual data from DataContainer
    data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]
    data['img'] = [img.data[0] for img in data['img']]

    # scatter to specified GPU
    data = scatter(data, [device])[0]

    img_metas = data["img_metas"][0]
    # forward the model
    with torch.no_grad():
        x = model.extract_feat(data['img'][0])
        proposal_list = model.rpn_head.simple_test_rpn(x, img_metas)

        img_shapes = tuple(meta['img_shape'] for meta in img_metas)
        scale_factors = tuple(meta['scale_factor'] for meta in img_metas)

        rois = torch.stack(proposal_list, dim=0)

        batch_index = torch.arange(
            rois.size(0), device=rois.device).float().view(-1, 1, 1).expand(
            rois.size(0), rois.size(1), 1)
        rois = torch.cat([batch_index, rois[..., :4]], dim=-1)
        batch_size = rois.shape[0]
        num_proposals_per_img = rois.shape[1]

        # Eliminate the batch dimension
        rois = rois.view(-1, 5)
        bbox_results = model.roi_head._bbox_forward(x, rois)
        cls_logits = bbox_results['cls_score']
        bbox_pred = bbox_results['bbox_pred']

        # Recover the batch dimension
        rois = rois.reshape(batch_size, num_proposals_per_img, -1)
	# logits
        cls_logits = cls_logits.reshape(batch_size, num_proposals_per_img, -1)
        bbox_pred = bbox_pred.reshape(batch_size, num_proposals_per_img, -1)

        scores = F.softmax(cls_logits, dim=-1)
        bboxes = model.roi_head.bbox_head.bbox_coder.decode(rois[..., 1:], bbox_pred, max_shape=img_shapes)
        # B, 1, bboxes.size(-1)
        scale_factor = bboxes.new_tensor(scale_factors).unsqueeze(1).repeat(
            1, 1,
            bboxes.size(-1) // 4)
        bboxes /= scale_factor

        det_bboxes = []
        det_labels = []
        det_probs = []
        for (bbox, score) in zip(bboxes, scores):
            det_bbox, det_label, det_prob = my_multiclass_nms(bbox, score,
                                                              cfg.model.test_cfg.rcnn.score_thr,
                                                              cfg.model.test_cfg.rcnn.nms,
                                                              cfg.model.test_cfg.rcnn.max_per_img)

            det_bboxes.append(det_bbox)
            det_labels.append(det_label)
            det_probs.append(det_prob)

    return det_bboxes, det_labels, det_probs

For more details and usage example, please check out from >>my gist here<<. Hope it helps!


docs: Add communication guidance to mentor guide.

Also link to GSoC's mentor guide; Outreachy doesn't really seem to have one:

Looks great to me!


Inconsistencies on when universal function is torn off

The following code is behaving differently on different platforms:
typedef T Foo<T>(T x);

T _foo<T>(T x) => x;

class Bar{
  final Foo f;
  const Bar._([this.f = _foo]);
  const Bar() : this._();
}

void main() {
  var b = new Bar();
  print(b.f.runtimeType);
}

On DDC and DDK, I see:
(dynamic) => dynamic

On Dart VM (--preview-dart-2):
<T>(T) => T

Older versions of the VM (2.0.0-dev.44.0) print (dynamic) => dynamic.
Dart2JS, otoh, gives a compile time error:
Error: Not a compile-time constant.
  const Bar._([this.f = _foo]);

The DDC behavior is correct.  Inference should instantiate the generic function with dynamic, and the result is a valid const value.

@a-siva @crelier - this looks like a recent regression in the VM.  Dev 44 VM was printing the right type.

@sigmundch @leafpetersen - dart2js should support this as well - is there already a bug for that?

@sjindel-google - type inference is done on the kernel side and this also involves partially instantiated functions. Could it be that runtimeType does not properly handles partially instantiated functions?

@crelier: It's possible -- it may also be related to the fact that the field initializer is being evaluated by the constant evaluator version of partial instantiation, not the regular (runtime) version.

I can handle the VM part of this.

dart2js now prints (dynamic) => dynamic


Add reconciliation timestamp to the Module CRD

Is your feature request related to a problem? Please describe.
There is currently reconciliation status on the Module CRD, but another useful information to add to the status would be to add the time when the reconciliation finished. Also, the time should be then propagated to the history entry.
Describe the solution you'd like
As a solution, we should add a new field to the reconciliation status and map it when setting the status of the module after reconciliation.

Hey @petar-cvit, I'm interested in this issue. Could you please assign it to me? 🙇🏻♂

@SpiffyEight77 do you have any updates on the issue?

@petar-cvit
I apologize for my delayed response. I've been extremely busy with work and will be until October, so I don't think I'll have enough time to complete this task. I've removed myself from the assignment.

@petar-cvit  I could work on this issue
Things needed to be done :

add a new field in the ReconciliationStatus struct for the HistoryEntry
Map the values of the status in the setStatus for the updated ReconciliationStatus

@Abiji-2020 yup, that sounds good. Just make sure that after you update the struct in go, you generate the manifests using make generate and make manifests and apply created manifests to your cluster

Hey @Abiji-2020, do you have any updates on the issue? Let me know if you have any questions

I have a doubt that currently before updating the code I tried to run the Makefile to generate but it is giving me nil pointer error and with no further details. So I am struck in that process

Hey @Abiji-2020, thanks for bringing this up.
There is a mismatch between go and controller-gen versions. I bumped the controller-gen version in #540. Sync your fork and pull the latest main branch.
To install a new version and generate manifests, run the following commands from the cyclops-ctrl folder.
Remove previous controller-gen:
rm -rf bin

Install the new version:
make controller-gen

and then you can generate code (make generate) and manifests (make manifests).

oh okay, I was a beginner so I was wondering whether I am doing anything wrong here

will update you after the correction


Very Useful - Picky request

This is by far the best progress bar to use in Jupyter, so thank you! I am being greedy now, would it be possible to implement an ETA based on previous iterations as well as keep track of running time at a certain interval. Not looking to get too crazy, just a basic average of time between task start and end multiplied by the number of tasks left. Then for the running time, I would imagine updating every second might be unnecessary so maybe an argument with a default set to something like 5 seconds.
Thank you again for sharing!

I took a stab at implementing this for times when is_iterator == False, seems to be working ok. There likely is a much better way to do this.
index = 0
avg_run = 0
time_elapsed = 0
iteration = 1
eta = 0
overall_start = time.time()
try:
	for index, record in enumerate(sequence, 1):
		start = time.time()
		if index == 1 or index % every == 0:
			if is_iterator:
				label.value = '{name}: {index} / ?'.format(
					name=name,
					index=index
				)
			else:
				progress.value = index
				label.value = u'{name}: {index} / {size} - Avg Time: {avg_run}s | ETA:{eta}s'.format(
					name=name,
					index=index,
					size=size,
					avg_run = '{0:.2f}'.format(avg_run),
					eta = '{0:.2f}'.format(eta)
				)
		yield record

		iteration_time = time.time() - start
		time_elapsed = time_elapsed + iteration_time
		avg_run = time_elapsed / iteration
		eta = (size - iteration) * avg_run
		iteration = iteration+1

except:
	progress.bar_style = 'danger'
	raise
else:
	progress.bar_style = 'success'
	progress.value = index
	label.value = "{name}: {index} - Total Time: {overall_time}".format(
		name=name,
		index=str(index or '?'),
		overall_time = '{0:.2f}'.format(time.time() - overall_start)

	)


feat(baseplugin): add internalfilesloader

Which problem is this PR solving?

Closes #216 Add internal files loader to BasePlugin implementation.

Short description of the changes

Resolves TODO of #158, which required this implementation. This PR moves the logic out of the grpc plugin to BasePlugin, in case future plugins also require this behavior.

ping, for more reviews.


Project status

Hi what is the status of the project? I have gone through both the papers AudioLM and Soundstream and it seems interesting, I would like to contribute to the project and learn along the way.

????

have a todo section now, you can track the progress there


Context path with multiple docker-compose files

I have an issue with this plugin.
I have multiple docker-compose files, and any of them have Dockerfile pathes in it. Furthermore, there are external links between them.
It seems that the first path declared in yml array will be the current path in context for all building and running docker-compose files.
I have two problems :

First one, the relative path to the Dockerfile will always start to the first yml file declared, so docker-compose can't find it. I solved temporarily the problem by setting absolute path, but this path isn't relevant outside my vagrant.
Second one, as docker-compose name built images with the parent directory of the docker-compose, all my built images will have wrong name, and all my external links are wrong too.

Sorry for my english, I hope that is understandable.

Let me echo back to make sure I understand the two issues:

The working directory for where to look for Dockerfiles is based off the first YML file, but should be configurable?
The names of the images built ues the parent directory, but should be configurable?

@leighmcculloch
I don't know of the challenges the OP had, but I can confirm issue nr. 1 not only for Dockerfiles, but .env too.
==> default: Couldn't find env file: /home/vagrant/projects/project1/.docker/wordpress/.env.local
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

 /usr/local/bin/docker-compose-1.24.1  -f "/home/vagrant/projects/project1/docker-compose.yml" -f "/home/vagrant/projects/project1/docker-compose.local.yml" -f "/home/vagrant/projects/project2/docker-compose.yml" -f "/home/vagrant/projects/project2/docker-compose.local.yml" rm --force

Stdout from the command:
Stderr from the command:

Couldn't find env file: /home/vagrant/projects/project1/.docker/wordpress/.env.local

The path .docker/wordpress/.env.local exists in project2.

Thanks for sharing an example. What do you think should change to make this work? Instead of passing each docker-compose file to a single call of docker-compose should each file be a separate execution of docker-compose?

Thats a good question ;-).
I am not deep enough inside docker-compose to tell if the call below
/usr/local/bin/docker-compose-1.24.1 -f "/home/vagrant/projects/project1/docker-compose.yml" -f "/home/vagrant/projects/project1/docker-compose.local.yml"
would be the same as
/usr/local/bin/docker-compose-1.24.1 -f "/home/vagrant/projects/project1/docker-compose.yml"
/usr/local/bin/docker-compose-1.24.1 -f "/home/vagrant/projects/project1/docker-compose.local.yml"
If this is the case is it only a matter of more memory consumption of the docker-compose executable? Or are we missing some other constraints which may occur?

What if you replace your one:
config.vm.provision :docker_compose, yml: ["project1/docker-compose.yml", "project2/docker-compose.yml"], run: "always"

With multiple:
config.vm.provision :docker_compose, yml: "project1/docker-compose.yml", run: "always"
config.vm.provision :docker_compose, yml: "project2/docker-compose.yml", run: "always"

Does that resolve the issue?

That did the trick. I just checked it.
Since I am using only docker-compose.yml files I am not noticing any naming issues on the containers.
Thank you :-).

Excellent. I'm going to close the issue. Anyone is welcome to reopen the issue if this is not addressed.


Update Spark version to 2.4.0 and make sure it is supported

This is a fix for https://datastax-oss.atlassian.net/browse/SPARKC-553.

Thanks for the fix! I'll try to quick off the tests

test this please

Jenkins can't hear me, i'll run them manually

test this please

Excellent tests are running

Build against Scala 2.10 finished with failure

Build against Scala 2.11 finished with failure

@nikolovivan looks like there is one more issue in the com.datastax.spark.connector.repl.CassandraRDDReplSpec
Another class not found. Fix up that test and I think we are good to go

Hey, I forgot that I had troubles last time running the integration tests and I had to run them from a clean Docker container. Just did it and I think I had to bump the Scala version to the same number they've used in Spark to fix the error.
I don't have the time to run the tests fully at the moment as I have to run, but feel free to trigger them again. If there are issues, I'll spend time properly addressing them over the weekend.

this this please

Hey, I forgot that I had troubles last time running the integration tests and I had to run them from a clean Docker container. Just did it and I think I had to bump the Scala version to the same number they've used in Spark to fix the error.
I don't have the time to run the tests fully at the moment as I have to run, but feel free to trigger them again. If there are issues, I'll spend time properly addressing them over the weekend.

No problem i'll trigger everything again

test this please

for some reason I have to say it twice ... or maybe since I was looking at the configuration it felt bad and decided to work the second time

Build against Scala 2.11 finished with success

Build against Scala 2.10 finished with failure

Looks like it's passing for 2.11, which is the only supported version since 2.3, so I guess it's good now.
Let me know if you would prefer me to see which could be the lowest possible version between 2.11.8 and 2.11.12 that would make the build pass?

I just checked and we could go as low as 2.11.9 if we wanted to. Might be best to keep it consistent with what version Spark 2.4.0 uses, though (2.11.12). Let me know what you prefer.

Does this also add support for scala 2.12?

No, it doesn't. Adding support for 2.12 might be trivial, but might also involve changing lots of other library versions and code.

2.10 Shouldn't pass on master (I just forgot to turn off triggering it) so This is good to go

Do you have an idea when you'll be merging this one and releasing a new version?

No, it doesn't. Adding support for 2.12 might be trivial, but might also involve changing lots of other library versions and code.

Yes.
Based on this, looks like scala 2.12 is going to be the default scala version going forward.

I've easily built the project against scala 2.12 but I had to disable the scoverage plugin.
Seems that enabling it requires upgrading sbt.

@RussellSpitzer do you have any idea when you're planning on merging this and releasing a new version. Let me know if I can help in any way.

any update here?

Just finished getting caught up at work, I'll do a release tonight.


Add eslint and fix problems

Clean things up so we can better enforce coding standards.

yay! eventually we probably want to axe jshint and make eslint part of appium-gulp-plugins, right?
👍

Yes to axing jshint. eslint is available through our plugins (just gulp eslint). It should eventually be in gulp lint, too, but that has to wait until all the packages support it.


Pygments version is too old

I was playing with this action and found a super weird error:
https://github.com/astrojl-test/lumache/runs/4251532795?check_suite_focus=true#step:3:186
 'HtmlFormatter' object has no attribute 'get_linenos_style_defs'

It looks like this method has been a part of pygments HtmlFormatter class since 2.7:
https://github.com/pygments/pygments/commit/728fd19ebcb69ddb2c7af159939e1a19ae53a892
However, I see this in the logs:
  Requirement already satisfied: Pygments>=2.0 in /usr/lib/python3/dist-packages (from sphinx~=4.2.0; extra == "doc"->lumache==0.1.0->-r /home/runner/work/lumache/lumache/./docs/source/requirements.txt (line 1)) (2.3.1)

I'm about to try upgrading it, but I'm anyway not quite sure how that ended up there.

This is a temporary project that might go away, apologies in advance if the links are broken at some point.

It is not issue of this action, and even not issue of Sphinx.
This error is throw by Furo, it requires pygments ~= 2.7 and you should add furo to your requirements.txt.

Furo is getting installed too. I then suspect that the pip version is too old, because it's not solving the dependencies: I see several errors like
  ERROR: nbconvert 6.3.0 has requirement pygments>=2.4.1, but you'll have pygments 2.3.1 which is incompatible.

I then suspect that the pip version is too old
Good point, it is possible bacause pip is installed from system package manager.

I want to known how do you install furo? I can found it in your requirements.txt.

The requirements file has a relative path pointing to this metadata file:
https://github.com/astrojl-test/lumache/blob/f0b705d799fd9c7740673aa9690637ff5147060c/pyproject.toml#L14

In page@v2, I prefer to used action/setup-python, then we can get a newer pip.

v2 has been published: https://github.com/sphinx-notes/pages/tree/v2
Feel free to reopen this if you still suffer this issue.


ROX-14148: Add transforming of declarative config to proto for Access Scopes

Description
This PR adds logic to transform a declarative configuration, specifically the Access Scope one, to its related proto message.
Within the transformation logic, the proto fields will be filled as well as a deterministic UUID based on the name of the access scope will be created.
For now, within the transformation logic, no validation of the transformed proto message is done (e.g. about conformity, name uniqueness etc.). These additional checks may be added at a latter point in a follow-up.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed

added unit tests.


Support C#

Hey, I love this tool. It would be great if it supported C#.
C# routes are often defined in the MVC construct. https://www.tutorialsteacher.com/mvc/routing-in-mvc
MVC routes would probably be the easiest pattern to identify and report on.
Additionally, routes can also be defined as web services exposed via ASMX or SVC files, or inside the web.config file.
.NET has the concept of HTTPModules which are triggered on every request, and also has the concept of IS API DLL extensions, which are typically executed when hitting the specific DLL file in the web root. All of these could be great to detect and easy to as well from reading the web.config file.

Hi @infosec-au
Thank you so much for your opinion!
I'll actively work on developing the aspects you mentioned to make them supported!

I started with MVC first. Since I'm not very familiar with C#, I can currently only identify URLs. I'll need to gradually improve from here.


Neuroml export

This should fix https://github.com/Neurosim-lab/netpyne/issues/266.
Minor change in pyNeuroML API...

This should prob not be merged until issue #268 mentioned here: https://github.com/Neurosim-lab/netpyne/pull/268/commits/2206852fb7cd4c9f8a055f13e68778fa7793d772 is fixed...


Completed event not fired on IOS

The Completed event is not fired on IOS, Safari keep navigate to the callback url without telling what happened.
The token cannot be catched.
public override void ViewDidAppear(bool animated)
{
    base.ViewDidAppear(animated);

    if (_isShown) return;
    _isShown = true;

    var auth = AuthenticationHelper.CreateOAuth2Authenticator(Provider);
    auth.Completed += OnAuthenticationCompleted;
    var view = auth.GetUI() as UINavigationController;
    PresentViewController(view, true, null);
}

async void OnAuthenticationCompleted(object sender, AuthenticatorCompletedEventArgs e)
{
    // NOT FIRED HERE
    if (e.IsAuthenticated)
    {
        var access_token = e.Account.Properties.FirstOrDefault(i => i.Key.Equals("access_token")).Value ?? string.Empty;
        await AuthenticationHelper.SocialCallback(Provider, access_token);
    }
    else
        await Navigator.Instance.CloseAuthentication();
}

Can you provide details? nuget version and how OAuth2Authenticator is constructed?
thanks

OAuth2Authenticator is constructed with this method:
public static OAuth2Authenticator CreateOAuth2Authenticator(Provider provider)
{
    OAuthConfiguration configuration =
        provider == Provider.Google ? OAuthGoogle :
        provider == Provider.Facebook ? OAuthFacebook :
        null;

    return new OAuth2Authenticator(configuration.ClientId, configuration.Scope, new Uri(configuration.AuthorizeUrl), new Uri(configuration.RedirectUrl));
}

public enum Provider
{
    Facebook,
    Google,
    Twitter
}

public static OAuthConfiguration OAuthGoogle
{
    get
    {
        return new OAuthConfiguration()
        {
            ClientId = "clientid.apps.googleusercontent.com",
            Scope = "https://www.googleapis.com/auth/userinfo.email",
            AuthorizeUrl = "https://accounts.google.com/o/oauth2/auth",
            RedirectUrl = "http://staging.logwod.me/social-callback/google",
        };
    }
}

public static OAuthConfiguration OAuthFacebook
{
    get
    {
        return new OAuthConfiguration()
        {
            ClientId = "clientid",
            Scope = "email",
            AuthorizeUrl = "https://www.facebook.com/dialog/oauth",
            RedirectUrl = "http://staging.logwod.me/social-callback/facebook",
        };
    }
}

The nuget version is <IP_ADDRESS>.
I've forgot to say that it works well on Android and UWP. The problem is only on IOS.

Will check tomorrow in the morning. This parts are going through some changes (WinRT is right now target and native UI - Chrome Custom Tabs and SFSafariViewController), so I might have messed something up.
So, UWP is working. Good news. Thanks.

@BenDevelopment
This iOS issue: was it on simulator? Even more precise Simulator for iOS 10.x?
Storing to KeyChain (Add method ) returns status code -34018 on all iOS 10.x simulators.
Investigating proper [re]solution/workaround.

Yes this is on simulator 10.2.
But the error you are describing here is not the same error. The "-34018" occure when you try to save data with the account factory. On IOS you must have a Entitlements.plist file to use local storage. But this is not this error I'm facing. My problem is juste that the event is not fired. I have no error message.

In 10.2 simulator I had ceashes with Xmarin.Auth.Authentication error and code -34018.
Switching to simulator 9.2 and 8.4 I was getting UIAlerts.
And it seems it is not only Entitlements.plist which needs to be added. It seems that XCode needs at list one entry and thus people add Key Chain Sharing in capabilities

Yes you have to add Entitlements.plist and add it to Custom Entitlements in the project settings. But it only works if you have a Apple provisionning profile (it doesn't works with the free developper account), that's why I 'm using a different tool to save datas in local storage.
If you have alerts with auth.GetUI() (I don't know if this is that you are talking about when you say UIAlerts), you have to add some modifications in the renderer (the "get started" guide seems to be out of date).
Try to replace:
PresentViewController (auth.GetUI (), true, null);
By:
UIViewController authView = auth.GetUI() as UIViewController;
PresentViewController(authView, true, null);

@BenDevelopment
I'm sorry, but I'm trying to answer bunch of questions and do some coding.

I'm using UIAlerts to show user token in Completed event handler.
This was what I was thinking with UIAlert
Component (and thus getting-started.md) is outdated. packaging component with Windows support and espacially samples is not trivial task anymore.

So this is new iOS API call:
    System.Object ui_controller_as_object = auth.GetUI();
    if (auth.IsUsingNativeUI == true)
    {
        SafariServices.SFSafariViewController c = null;
        c = (SafariServices.SFSafariViewController)ui_controller_as_object;
        PresentViewController(c, true, null);
    }
    else
    {
        UIViewController c = (UIViewController)ui_controller_as_object;
        PresentViewController(c, true, null);
    }

Yes it is breaking change System.Object is returned!!
And here I subscribe to event
https://github.com/xamarin/Xamarin.Auth/blob/portable-bait-and-switch/samples/Traditional.Standard/references01projects/Providers/Xamarin.Auth.Sample.XamarinIOS/TestProvidersController.cs#L91
And handle event (here be UIAlerts)
https://github.com/xamarin/Xamarin.Auth/blob/portable-bait-and-switch/samples/Traditional.Standard/references01projects/Providers/Xamarin.Auth.Sample.XamarinIOS/TestProvidersController.cs#L166-L244
See this issue with more details for NEW API.
https://github.com/xamarin/Xamarin.Auth/issues/146#issuecomment-284730760

I can confirm that it seems like the completed eventhandler is NOT called when the browser is redirected to redirectUrl, using version <IP_ADDRESS> and iOS 10.2 simulator.
I have the following code:
						auth.AllowCancel = true; 						auth.IsUsingNativeUI = true; 						auth.Completed += completedHandler; 						var ui = auth.GetUI(); 						PresentViewController ((SafariServices.SFSafariViewController)ui, true, null);
but the completedHandler is not called. The view just shows the redirectUrl, but no event is fired. There are no errors reported, since I do not save anything to the keychain.

Same for me. Completed-event does not fire on iOS, tried Simulator iPhone 6 iOS 9.1, Iphone 5s iOS 9.1, iPhone 7 iOS 10.2 and in none of them is the event fired.
Tried with both:
PresentViewController((SafariServices.SFSafariViewController)oauth.GetUI(), true, null);
and
PresentViewController((UIViewController)oauth.GetUI(), true, null);
Thanks!

Any idea  of how to fix this ?

@BenDevelopment @DFerenczi @mkroghj
So, my tests iOS != 10.x, nuget <IP_ADDRESS>
(abandoned search for 10.x issues until new features are implemented)

FaceBook WWW type app Redirect URL http[s]://xamarin.com
OK
FaceBook WWW type app Redirect URL http[s]://localhost
OK (getting access_token with error trying to load localhost!)
FaceBook WWW type app Redirect URL http[s]://<IP_ADDRESS>
OK (getting access_token with error trying to load localhost!)
github app Redirect URL http://xamarin.com
OK
github app Redirect URL https://xamarin.com
OK
github app Redirect URL https://localhost
OK
github app Redirect URL https://<IP_ADDRESS>
OK

NOTE:

support for hosts like localhost and <IP_ADDRESS> is NEW
NOT using custom url scheme[s]! This was never supported in Xamarin.Auth and this is under heavy development right now. Thus SFSafariViewController will NOT work (it might if you are lucky, but I would not bet).

@moljac Sorry, I may be misunderstanding - so you are not testing on 10.x until new features are added? I think Apple announced in January, that over 75 % of all of their active devices are now using iOS 10. Seems to me that this should be prioritised.

@moljac We have a finished iOS app, ready to be put into App Store - except for users not being able to authenticate because of the bug reported by BenDevelopment.
I agree with mkroghj: as just about 80% (as of February 20th this year, see https://developer.apple.com/support/app-store/) of iOS users now run iOS 10.x this is a major bug.
Hopefully you'll be able to squeeze out the bug very soon. Have you got a time estimate?

The functionality of Xamarin.Auth <IP_ADDRESS> is not fully implemented. The best course of action is to downgrade to <IP_ADDRESS> (the last stable before new partial functionality made it into the package)
The changes being made refer to the Google Authentication changes happening on 20th April 2017 (https://developers.googleblog.com/2016/08/modernizing-oauth-interactions-in-native-apps.html)
Hopefully the new features will be released by then, but if you can't wait, downgrade to <IP_ADDRESS> and use the old authentication method. Other than that there is nothing you can do until the functionality is finished.

@Jake-Barrow
Users can not use this functionality by default (implicitly) and especially should not for production.
This features are not turned on (yet), but they will be.
User needs to explicitly turn on Native UI in constructor to obtain SFSafariViewController and Custom Tabs.
New packages have this for testing, but it is undocumented and unstable. Old docs for Xamarin.Auth component are still valid except the cast and nuget dependency on Android.Support.Customtabs and this dependency will be removed (Android only).
And now how did you deduce that this bug is about new feature?

Fixed in <IP_ADDRESS>.
My mistake and I sincerely apologize. All this hectic with new Windows support and NativeUI made me push wrong logic (new NativeUI support turned on in the library). Those users that were using explicit constructor parameter to turn off NativeUI did not have issues.
Please let me know the results.
Now some details:
@mkroghj

@moljac Sorry, I may be misunderstanding -

Yes. You misunderstood: those tests did not cover iOS 10.x simulator on that day (Thursday last week). Explanation why I do not use iOS 10.x on daily basis is below.

so you are not testing on 10.x until new features are added?

I'm testing 10.x on device and to keep order in my life - iOS device tests beginning of the week. Why because I'm covering iOS, Android and several Windows Platforms right now and if I would go and redo all test on all platforms both simulators/emulators and devices I wouldn't write single line of new code.
The reason for not testing iOS 10.x on simulator is simply I'm experiencing (in most, but not all cases) issues described here:

139 Could not save account to KeyChain: -34018
https://github.com/xamarin/Xamarin.Auth/issues/139
128 Error: Could not save account to KeyChain -- iOS 10
https://github.com/xamarin/Xamarin.Auth/issues/128

Issue comment which shows that we have to help users with really basic things:
https://github.com/xamarin/Xamarin.Auth/issues/128#issuecomment-285351561
So, 2 issues for the same bug and bug which has NOTHING to do with Xamarin. To prove that few links from Apple developer forums, stackoverflow and some products hit by this bug (hockeyapp):
https://forums.developer.apple.com/message/179846
https://forums.developer.apple.com/thread/60617
https://forums.developer.apple.com/thread/51071
https://forums.developer.apple.com/thread/62847
http://stackoverflow.com/questions/38456471/secitemadd-always-returns-error-34018-in-xcode-8-in-ios-10-simulator
https://support.hockeyapp.net/kb/client-integration-ios-mac-os-x-tvos/known-issues-with-ios#hockeysdk-ios-and-ios-10-simulators
https://support.hockeyapp.net/discussions/problems/63710-the-authentication-token-could-not-be-stored-due-to-a-keychain-error
So, I have added exception that in Message explains user to add Entitlements.plist, but this is semi-workaround, because adding Entitlements just for simulator causes problems for device. (Read apple devs' opinions). So, I still have no solution for iOS 10.x simulator and I have XCode 8.2, so the problem is not fixed yet.

I think Apple announced in January, that over 75 % of all of their active devices are now using
iOS 10.

OK. But quoting numbers does not increase my opinion on the iOS 10 importance. I'm aware of it.

Seems to me that this should be prioritised.

So, today is Monday and I just tested <IP_ADDRESS> on 2 iOS 10.x devices and I was able to get token from Facebook even for http[s]://<IP_ADDRESS> and http[s]://localhost redirect_urls. Yes I get OnError first that UIWebView cannot connect to server (of course), but OnError event is triggered and after that onComplete.
In <IP_ADDRESS> which is  published, this is fixed so no error is raised. I was using explicit constructor parameter and people that read my samples could deduce and turn old API on (and new off).
My mistake.
@rasweh

@moljac We have a finished iOS app, ready to be put into App Store - except for users not being
able to authenticate

OK. 1st I'm sorry that you are/were blocked. Please test <IP_ADDRESS> and report if this fixes the issue.

Hopefully you'll be able to squeeze out the bug very soon.

Nuget <IP_ADDRESS> out since yesterday. Please report outcome/results.

Have you got a time estimate?

For new API (NativeUI support)?
I wish. I was afraid that Windows (WinRT both Windows Store and Windows Phone, UWP, Windows Phone 8.x Silverlight) support will bring influx of people to Xamarin.Auth and thus more issues and feature requests. And this happened and now I'm a bit under pressure on all fronts.
Generally my priority list:

NativeUI support ASAP alpha of usable API for testing
fix few issues in Android (most of them not critical)
component packaging (including docs)
Windows support (few nice-to-haves)
new features (some ideas like fingerprint etc)

I am using the <IP_ADDRESS> and the auth completed is not fired for Android,it just redirects to url and doesnot start the activity

@sopoco @roopeshnaik
I'm using <IP_ADDRESS> and OAuth2 events for

Facebook Android and iOS
Microsoft Live Android and iOS
and
Github (Explicit) iOS only (Android fires OnError)

are firing.
Twitter OAuth1 events are not firing and I'm working on that.
Current work in progress:

fixing Android code for OAuth1 and loops in OnError
cleaning Android code and adding Xamarin.Auth.Exceptions for errors with messages
native support

how can i handle the redirect uri from the custom chrome tabs
        auth.Completed += Auth_Completed;

        var ui_intent = auth.GetUI(this);
        if (auth.IsUsingNativeUI == true)
        {
            // using new Native UI API Chrome Custom Tabs on Android
            Uri uri = auth.GetInitialUrlAsync().Result;
            Android.Net.Uri uri_android = Android.Net.Uri.Parse(uri.AbsoluteUri);

            CustomTabsActivityManager customTabs = new CustomTabsActivityManager(this);
            CustomTabsIntent cti = null;
            cti = (CustomTabsIntent)ui_intent;
            cti.LaunchUrl(this, uri_android);
        }

After authenticating it redirects to uri and do i need to Startactivity after cti.LaunchUrl(this, uri_android)

@roopeshnaik The NativeUI support is not yet done, so no events are fired. For now you need to use the old UI (IsUsingNativeUI = false in ctor)

@Jake-Barrow , the Old Webviews are supposed to be deprecated/blocked by 20-04-2017, so this method will not work for Google Oauth2
https://developers.googleblog.com/2016/08/modernizing-oauth-interactions-in-native-apps.html

@roopeshnaik
Could you be so kind and read threads/issues a bit before posting? Please.
I am (we are) aware of those facts and are working on solution.
See my weekend work repo:
https://github.com/moljac/HolisticWare.Android.Support.CustomTabs.Chromium.SharedUtilities
This is being silently integrated added, so users can use Native OAuth + apply style and themes.
Instead of just stating "X is not working" provide a little bit more info.

Nice to see that the completed callback is working again! GJ 👍 I don't know if this problem is somehow related to this, please tell me if I should create a new issue, but im always getting a popup saying "Error authenticating: invalid_request" even though the fetch is successful i.e. I get a valid access token.
Any ideas?
/Ferenczi

@DFerenczi
It was working all the time, but users have turned on new API which is not completely finished. I tried to prepare more flexible custom schemes support, but this will not be possible (http and https in Custom Tabs and SFSafariViewController). Good thing is that new API is close to be finished.
Regarding your problem: I need more info. I presume this is Android. Am I right? I need to clean up some code it seems that it runs into error loop, but not 100% sure.
Which version of nuget? Which provider and OAuth protocol?

Sound good!
I get the error msg on both platforms (ios and android), i'm using <IP_ADDRESS> on iOS and <IP_ADDRESS> on Android while using OAuth2Authenticator.. I've tried to set the ShowErrors property to false just to avoid the popup but it still shows. Hope this helps in any way.

the event doesnt fired in the new version please fix it

@ads90

which is new version?
can we see your ctor? (Are you using nativeUI)

iam using xamarin auth 1.4.1,the completed event is not getting fired after the navigation to redirection url.

hi please see the details of constructor  and safari navigation
auth2Request = new OAuth2Authenticator(
clientId: clientIdGoogle, // Oauth2Request client id
scope: scopeGoogle, // scopes
authorizeUrl: new Uri(authorizeUrlGoogle), // the authorization URL f
isUsingNativeUI: true,
redirectUrl: new Uri(redirectUrlGoogle)); // the redirect URL for
nd safari navigation
SafariServices.SFSafariViewController c = null;
c = (SafariServices.SFSafariViewController)ui_controller_as_object;
					if (UIDevice.CurrentDevice.CheckSystemVersion(10, 0))
					{
						c.PreferredBarTintColor = UIColor.Gray;
						c.PreferredControlTintColor = UIColor.White;
					}
					else
					{
						c.View.TintColor = UIColor.Gray;
					}

					Action view_controller_customization =
						() =>
						{
							//c.NavigationController.NavigationBar.TintColor = color_xamarin_blue;
						};
					//------------------------------------------------------------
					// [REQUIRED] launching SFSafariViewController
					PresentViewController(c, true, view_controller_customization);
					//=================================================================

@moljac I too face the issue that @githubdigo is faced with. Sorry to ask, but is there something (possibly trivial) that we are missing in common?

@JichinThomas @githubdigo
Can you please provide:

AppDelegate.OpenUrl code
Info.plist and
redirect_url

AppDelegate OpenUrl code
Public override bool OpenUrl ( UIApplication application,NSUrl url,string sourceApplication, NSObject annotation){
System.Text.StringBuilder sb = new System.Text.StringBuilder();
sb.AppendLine("OpenURL Called");
sb.Append("     url         = ").AppendLine(url.AbsoluteUrl.ToString());
sb.Append("     application = ").AppendLine(sourceApplication);

sb.Append("     annotation  = ").AppendLine(annotation?.ToString());
System.Diagnostics.Debug.WriteLine(sb.ToString());

System.Uri uri = new Uri(url.AbsoluteString);
IDictionary<string, string> fragment = Utilities.WebEx.FormDecode(uri.Fragment);
Account account = new Account(
                        "username",
                         new Dictionary<string, string>(fragment) );
return true;

}
Info.plist
CFBundleIdentifier     com.xyz.candidate.qa
CFBundleURLName     com.xyz.candidate.qa
CFBundleURLSchemes
xamarin-auth
xamarinauth
xamarin.auth
CFBundleURLTypes     Viewer
Redirect URL used is http://xamarin.com

@JichinThomas
redirect_url!!! NO GOOD!
iOS will intercept http redirect uri in Browser, so no events will be fired.
It must be:
xamarin-auth

or
xamarinauth

or
xamarinauth

And those are Xamarin.Auth schemes - you need to come up with your own!
And one more question: Which OAuth provider?

Also, am getting an alert when trying to authenticate using Xamarin.Auth2.  Have you seen a similar one when trying Xamarin.Auth2 Native UI case ?

@moljac  Am using Google Sign in option using Xamarin.Auth2 1.4.1  version

cannot see image?
Is it something like:
            if (scheme.StartsWith("http", StringComparison.InvariantCultureIgnoreCase))
            {
                StringBuilder sb = new StringBuilder();
                sb.AppendLine("WARNING");
                sb.AppendLine($"Scheme = {scheme}");
                sb.AppendLine($"Native UI used with http[s] schema!");
                sb.AppendLine($"Redirect URL will be loaded in Native UI!");
                sb.AppendLine($"OAuth Data parsing might fail!");

                ShowErrorForNativeUI(sb.ToString());
            }

As you can see I did put this WARNING for users to WARN them that they are using http or https schema with SFSafariViewController and that Redirect URL will NOT be loaded and OAuth data will NOT be parsed!!!!!
For google sign in this should be YOUR (AND ONLY YOURS) redirect_url
com.xyz.candidate.qa:/oauth2redirect

@moljac
yes, this is the warning.
Is the second part in the redirect url, the callback function ?

@JichinThomas Sorry, but I do not understand "second part"!
If you need faster support go to slack Xamarin Community chat in #xamarin-auth-social channel and ping us there. There are several people that have implemented new OAuth with google and other providers.
Using github issues as chat is at least overkill.

+1

@biapar ??

@moljac: the events are not intercepted with native flag on ( see Google reauirements ). I use xamarin firm and todoawsauth. Facebook auth is ok. Xam.auth lib 1.4.1

@biapar
I bet your redirect-url is http://* or https://*.
You need custom scheme (registered), so your app intercepts opening page with custom scheme and parses url and raises events.
+1 does not help us at all. You need to provide more info if you want to be helped out, otherwise I must resort to ignore mode.

I redirect to my site http://. How to do the "custom scheme"?

@biapar

I redirect to my site http://

Another bet won!
This is exactly your problem - NativeUI CANNOT work with http[s] schemes!
OK. It might/could, but with a lot of effort and no guarantees for success.

How to do the "custom scheme"?

Sorry too long for answer.
Check:
https://github.com/moljac/Xamarin.Auth.Samples.NugetReferences/blob/master/Xamarin.Forms/Evolve16Labs/Portable/MainPage.xaml.cs#L406
and
https://github.com/moljac/Xamarin.Auth.Samples.NugetReferences/blob/master/Xamarin.Forms/Evolve16Labs/Portable/MainPage.xaml.cs#L412
Custom scheme is anything that is not here:
https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml

Thank you. Why Do you use different redirect URI for IOS and Android?

What type of client OAUTH Do you created into Google Console? I created web application.

@biapar

Thank you. Why Do you use different redirect URI for IOS and Android?

Good. Very good question.
Do you remember creating Credentials for Android and iOS app?
You had to submit Package ID on Android (+ SHA) and Bundle Id on iOS. OK. Some developers could make those match, but most of the IDEs (XCode, Android Studio, Xamarin Studio, Visual Studio) will make it based on project settings - project name user defines during project creation.
Based on Project Id and/or Bundle Id google creates schema for callback url. Why? My speculation is to track your app[s].

What type of client OAUTH Do you created into Google Console?

Android, iOS Other.
Those are (according to google) Installed apps.

I created web application.

No. Wrong. Web App is let say for server (ASP.net app that has login with google. Saving secure data there is considered OK (debatable), but on mobile it is not.
Xamarin.Auth used to work with Web Apps only (needed real redirect-url, not localhost,or custom schemes), but this has changed

Closing.

So for Google Sign In using this instructional https://developer.xamarin.com/samples/xamarin-forms/WebServices/OAuthNativeFlow/ takes me to google sign in, once I sign in my browser stays open and the OnAuthCompleted event never get's fired. I set the breakpoint and it never gets there. So things seem to be working to an extent.. Just need this OnAuthCompleted event to fire off.
`		void OnLoginClicked(object sender, EventArgs e)
{
string clientId = null;
string redirectUri = null;
		switch (Device.RuntimePlatform)
		{
			case Device.iOS:
				clientId = Constants.iOSClientId;
				redirectUri = Constants.iOSRedirectUrl;
				break;

			case Device.Android:
				clientId = Constants.AndroidClientId;
				redirectUri = Constants.AndroidRedirectUrl;
				break;
		}

		var authenticator = new OAuth2Authenticator(
			clientId,
			null,
			Constants.Scope,
			new Uri(Constants.AuthorizeUrl),
			new Uri(redirectUri),
			new Uri(Constants.AccessTokenUrl),
			null,
			true);

		authenticator.Completed += OnAuthCompleted;
		authenticator.Error += OnAuthError;

		AuthenticationState.Authenticator = authenticator;

		var presenter = new Xamarin.Auth.Presenters.OAuthLoginPresenter();
		presenter.Login(authenticator);
	}

	async void OnAuthCompleted(object sender, AuthenticatorCompletedEventArgs e)
	{
		var authenticator = sender as OAuth2Authenticator;
		if (authenticator != null)
		{
			authenticator.Completed -= OnAuthCompleted;
			authenticator.Error -= OnAuthError;
		}

		User user = null;
		if (e.IsAuthenticated)
		{
			// If the user is authenticated, request their basic user data from Google
			// UserInfoUrl = https://www.googleapis.com/oauth2/v2/userinfo
			var request = new OAuth2Request("GET", new Uri(Constants.UserInfoUrl), null, e.Account);
			var response = await request.GetResponseAsync();
			if (response != null)
			{
				// Deserialize the data and store it in the account store
				// The users email address will be used to identify data in SimpleDB
				string userJson = await response.GetResponseTextAsync();
				user = JsonConvert.DeserializeObject<User>(userJson);
			}

			if (account != null)
			{
				store.Delete(account, Constants.AppName);
			}

			await store.SaveAsync(account = e.Account, Constants.AppName);
			await DisplayAlert("Email address", user.Email, "OK");
		}
	}

	void OnAuthError(object sender, AuthenticatorErrorEventArgs e)
	{
		var authenticator = sender as OAuth2Authenticator;
		if (authenticator != null)
		{
			authenticator.Completed -= OnAuthCompleted;
			authenticator.Error -= OnAuthError;
		}

		Debug.WriteLine("Authentication error: " + e.Message);
	}`

I'm facing the same issue that @Bmahecha123 is facing.

For those who are facing an issue not being redirect to their app after the authentication, the docs is actually missing an essential piece of code. Here's the link: https://github.com/xamarin/Xamarin.Auth/blob/master/samples/Xamarin.Forms/Evolve16Labs/05-OAuth/iOS/Xamarin.Auth/AppDelegate.OpenUrl.cs
The AuthenticationState is an instance of OAuth2Authenticator.

I found a solution that hasn't been mentioned:
Setting the HttpClient implementation to NsUrlSession instead of Managed in the Project Options -> iOS build.
Managed is slower, doesn't support TLS 1.2 (needed for ATS, set as default on iOS9+).
The only downside to NSUrlSession is that it requires iOS7+. There's another client that requires iOS6+.
Before this, auth.Completed mostly fired right away but on every 2nd or 3rd attempt it took over a minute, while the user was stuck in the browser, logged into google, on the google start page.
More Details here:
https://stackoverflow.com/a/57875600/10295942

Hello Everyone , Anybody can help me out that I am getting attached screen shot warning on last line of below code (presentor) I am testing this on simulator i.e on IOS. I have sent all the required parameters.

var authenticator = new OAuth2Authenticator(
clientId: clientId,
clientSecret: string.Empty,
scope: Constants.FacebookScope,
authorizeUrl: new Uri(Constants.FacebookAuthorizeUrl),
redirectUrl: new Uri(redirectUri),
accessTokenUrl: new Uri(Constants.FacebookAccessTokenUrl),
getUsernameAsync: null,
isUsingNativeUI: true
);
authenticator.Completed += OnAuthCompleted;
authenticator.Error += OnAuthError;
                //authenticator.IsLoadableRedirectUri = true;

                AuthenticationState.Authenticator = authenticator;

                var presenter = new Xamarin.Auth.Presenters.OAuthLoginPresenter();
                **presenter.Login(authenticator);**


Add key_value function

It would be nice to be able to use names as keys a bit more intuitively than we can now. The difficulty in using keys as names crops up in maps: no way to access names without mapping the names as well:
Map(function(x, x_name) str_c(x, x_name), y, names(y))

A similar issues crops up with loops where we are forced to use seq_along or some kind of indexing to access the name property.
The alternative would be to re-define the vector / list to be a new list with named key / values. We could then do something like this:
Map(function(x) str_c(x$value, x$key), y))

An initial implementation may look something like this:
key_value <- function(lst) {
  kv <- Map(function(x, y) list(key = x, value = y), names(lst), lst)
  names(kv) <- NULL
  kv
}

Fixed by https://github.com/Ottawa-EPRI/EPmisc/commit/3f0f326d4e045bd35894c28c0bf0efcd9c07952f


deep model won't work

I'm submitting a bug

What is the current behavior?

the app won't work with deep model passing to label or input

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem via

this is the version without bug
https://jsfiddle.net/9ody3m2g/
this is a version with the bug
https://jsfiddle.net/1bo0pvad/1/

What is the expected behavior?
showing the data from a model in the input

Please tell us about your environment:

Version: 2.3.1
Browser: all

Fixed in #535