empire suddenly stopped reading contents of procfile

More to come on this as I get more info today, but we got hit by this pretty hard this.
Our version of empire client and empire controller hasn't changed in ~6 months (I'll get the version of both shortly).
As of a few days ago our emp deploys started failing.
Emp deploys started failing even with docker images that are happily running in the cluster.
The symptoms are this: the command in the procfile is being ignored.  I verified that the ECS task definition that empire is creating/updating for the service has "python2", not the shell script that is in the procfile.
My only explanation is that something recently changed in ECS that is preventing empire in updating the run command.
Our workaround is to re-build every single one of our docker images to include CMD in the Dockerfile to match that of the procfile.

A couple things:

What's the version of emp/empire?
Can you share the contents of the Procfile?
Can you share the contents of the Dockerfile?
How many apps are you working with?  Did it break for all of them?

Thanks!

 It broke for 2 of them.
We use terraform to deploy empire (ie an RDS database, 2 empire controllers behind an ELB)

Some more notes:

I competely started from scratch. I completely destroyed the empire RDS database, deleted the ECS cluster, and de-registered all 15K tasks (since there is now way to delete tasks) to start fresh.  There is now only the "default" ECS cluster.
I then recreated the empire RDS database, created our "stage" cluster, and ran "emp deploy openresty-router"
The openresty-router app worked!
I then deployed the next app, that was originally causing problems, and the problem persists: It is running "/bin/bash" as the command, NOT /code/empire-run.sh:

[root@ip-10-20-2-175 ec2-user]# docker ps -a
CONTAINER ID        IMAGE                            COMMAND             CREATED              STATUS                          PORTS               NAMES
00b3b0f3caae        clearcare/web:1450a0d            "/bin/bash"         5 seconds ago        Exited (0) 4 seconds ago                            ecs-0ea7f12c-8a45-4ac8-a419-38433a37baa4--web-2-web-84ae92cbafe6a0810400

I ended up bashing onto this stopped container like so, just to snoop around (procfile is in the correct directory, along with the script that it usually runs):
docker commit 00b3b0f3caae foo
docker run --entrypoint=bash -it foo
# ls | grep Procfile
Procfile
# cat Procfile
web:                 /code/empire-run.sh
# pwd
/code

So in a nutshell, it appears the WORKDIR is correct, and the Procfile is there.
I'm just not sure why it works on one app, but no this one, and we are almost 100% certain this image  was working fine just last week.
We are looking at the extractor.go function and see what could be tripping it up.

Oh, and no errors show up in the empire logs when creating the app:
017-01-22T20:57:55+0000 lvl=info msg=request.complete request_id=
t=2017-01-22T20:58:04+0000 lvl=info msg=PullImage request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e registry= repository=clearcare/web tag=1450a0d
t=2017-01-22T20:58:05+0000 lvl=info msg=ListServicesPages request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e
t=2017-01-22T20:58:06+0000 lvl=info msg="creating load balancer" request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e err=nil name=1383bbcc8ed0477196907019b760d9bb external=false dns-name=internal-1383bbcc8ed0477196907019b760d9bb-593636305.us-west-2.elb.amazonaws.com cert=
t=2017-01-22T20:58:08+0000 lvl=info msg=RegisterTaskDefinition request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e family=2e6d80dd-dc4a-4104-92e6-092f5965bc99--web
t=2017-01-22T20:58:08+0000 lvl=info msg=UpdateService request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e service-name=2e6d80dd-dc4a-4104-92e6-092f5965bc99--web desired-count=1 task-definition=2e6d80dd-dc4a-4104-92e6-092f5965bc99--web
t=2017-01-22T20:58:08+0000 lvl=info msg=CreateService request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e service-name=2e6d80dd-dc4a-4104-92e6-092f5965bc99--web desired-count=1 task-definition=2e6d80dd-dc4a-4104-92e6-092f5965bc99--web
t=2017-01-22T20:58:08+0000 lvl=info msg=request.complete request_id=140b25a9-ce84-4c54-9ffc-ab5a5d3dc99e

I then deployed the EXACT image that is working just fine in our AWS prod account to our newly created cluster and it failed!  There has to be some unknown difference that I am overlooking.
Looking at the extractor.go file, it looks like the empire controller downloads the image, creates a container, and then extracts the contents of the procFile.  I'm going to investigate that area to see if there is something funny going on there.

Update: I may have found the breaking difference:
Broken:
stage empire: Docker version 1.12.6
stage ecs: Docker version 1.12.6
Works
prod empire: Docker version 1.11.2
prod ecs: Docker version 1.12.6
I'm going to investigate this.  It appears docker version 1.12.6 is breaking the empire controllers.

@grayaii Curious - I'd be interested to figure out why 1.12.6 is working for some of your apps, but not others.  Did you explicitly make the change to 1.12.6?
I know that other folks (in the Slack) have had issues going to 1.12.6, and we haven't done any testing with it.  Be curious to hear what you find.

@phobologic Ah ha! Everything went back to working once I downgraded the empire controllers to 1.11.2!!!  I'm guessing that the 1.12.6 version of docker packs things differently in the image/tar file and thus empire's extractor.go file does not find the procfile and falls back to using CMD.  We upgraded the version of docker in our ECS cluster a few weeks ago and we had no issues, but I did not know we upgraded the version of docker on the empire controllers, which was recently done.
I see that the version of empire that we are using is very old and the new code of empire uses a completely different way of extracting the contents of the procfile.  I'll see if we can merge that code into your local branch to fix this (I have no idea how long AWS will keep 1.11.2 in their repos, so it will good for us to fix it).
I have no idea why some apps work while others don't.  All of our Jenkins slaves that build the actual images are using 1.11.2 too. That hasn't changed.
Whew!  Once we get our empire controllers working with 1.12.6 on our end, I'll let you know what we fixed.  Good to know that other people are are feeling the pain.  We had to update to 1.12.6 in our ECS cluster because we had to update our ECS agent, and the version that we upgraded to was failing with docker 1.11.2 and thus we had to update to 1.12.6.
Feel free to close this ticket, but if we fix it, I'll definitely share the fix here!!!

@grayaii Ah, ok - yeah, docker doesn't keep it's API steady between minor releases, so I'm not surprised there have been changes.  Thanks for digging into it, and glad you were able to find a work around.  Please keep us updated with what you find!


Reintroduce persistent local contracts deployment loaded at runtime

See https://github.com/gnosis/gp-v2-services/pull/1371 .

Do you think it maybe makes sense to have:
struct Contracts {
   // all deployed contracts
}

Yes. See #1379 above.That fact that I would have broken staging with the other PR is a little scary so now I'm thinking I should do that issue first.


Support custom RoleArn in Athena connection URI

This patch allows to provide a custom RoleArn in Athena Connection URI. This is useful when we need to switch to another role from within Ec2 instance profile.
Example Connection Uri:
awsathena+jdbc://athena.ap-southeast-1.amazonaws.com:443/testing?s3_staging_dir=s3://athena-query-results-testing&role_arn=arn:aws:iam::024150960000:role/AthenaPowerUsers

cc: #57
Hi @laughingman7743 ,
I'm using superset (https://github.com/apache/incubator-superset) which uses PyAthenaJDBC 1.2.x. The superset instance is using Ec2 IAM profile to get session tokens, however we want to work on Athena from another AWS account, and we think that using AssumeRole is better than providing credential file and/or fixed IAM credentials.
I think that the Java library doesn't support role switching (I did take a look at their documentation downloaded from here 1. That's why I try to have a work-around from your Python library. As I don't want to hurt it so much I create something new session as you can see in this PR; clearly the better way is to support RoleArn argument under your session token generating (https://github.com/laughingman7743/PyAthenaJDBC/blob/master/pyathenajdbc/connection.py#L46).
That's to say. I like to hear your advice how to support RoleArn and how to improve this PR. I'm eager to provide some support and/or create a new one if necessary.
Thanks a lot

Thanks PR. But I hope you will use PyAthena (https://github.com/laughingman7743/PyAthena). Since it supports SQLAlchemy, it can also be used in Superset. The scheme of URI is awsathena + rest.
https://github.com/laughingman7743/PyAthena#sqlalchemy
PyAthena supports boto3 authentication.
http://boto3.readthedocs.io/en/latest/guide/configuration.html
AssumeRole is also supposed to work if you create a configuration file.
http://boto3.readthedocs.io/en/latest/guide/configuration.html#assume-role-provider
Thanks,

Thanks a lot @laughingman7743 . I will take a look of using PyAthena in Superset.
I don't really want to use AssumeRole in file configuration (that would require us to provide IAM credentials in configuration too.) What we need is to get AssumeRole after getting session tokens from EC2 instance profile. I will take a look if there is a way.
Thanks again,

If you create a configuration file that specifies only role_arn and role_session_name, it may be that the action of AssumeRole is executed in the role attached to the EC2 instance? However, it may be useful to be able to specify a role_arn with a URI.
Try out the configuration file with PyAthena for the time being. If there is a problem, please raise an issue to the PyAthena repository (https://github.com/laughingman7743/PyAthena/issues).

Hi @laughingman7743 ,
How did you mention that superset could use PyAthena library? From the official documentation here
https://superset.apache.org/installation.html#deeper-sqlalchemy-integration
the only Athena URI can be read is awsathena+jdbc://....
Thanks a lot.

I do not use Superset so I do not know the details.
Perhaps I think installing the library with pip install PyAthena and specifying the URI starting with awsathena+rest:// in the SQLAlchemy URI.

Hi @laughingman7743, that's perfect. However I still need to create local ~/.aws/config configuration files. Do you think that we may have some option to provide rolearn in URI? If that's fine I would create new ticket on the ticket there

I tried implementing it. Please check the following branch.
https://github.com/laughingman7743/PyAthena/pull/40

That's great. I will close this ticket and use PyAthena instead. Thanks again for your support.


for falco add-on

Hi
Enable n disable add-on scripts are added for  falco and falcosidekick through helm3 . With this add-on user can enable audit-logs in microk8s. falco n falcosidekick extracts in a json format from any anomalies for further debugging

Hi
Since its not a draft request, i am closing this and will raise a new PR

Hi
Since its not a draft request, i am closing this and will raise a new PR


Add shebang to examples

Hey @frafra - these files are not set as executable (mode 644), so adding the hashbang won't make them executable.  I think it would either make sense to keep them as python files (.py, and run as python filename.py) or make them executable by changing the file mode and removing the .py if the hashbang is added.

Hey @frafra - these files are not set as executable (mode 644), so adding the hashbang won't make them executable.  I think it would either make sense to keep them as python files (.py, and run as python filename.py) or make them executable by changing the file mode and removing the .py if the hashbang is added.

Hey @bmuller, thank you for your reply.
I made them executable, but I am not sure on renaming them; it is still valid to keep the extension, as some editors rely on file extensions, but if you are sure about that I can drop it.

Hey @bmuller, thank you for your reply.
I made them executable, but I am not sure on renaming them; it is still valid to keep the extension, as some editors rely on file extensions, but if you are sure about that I can drop it.

Hey @frafra, I've thought about this some more, and I think the best thing to do would be to leave these as .py files that are not executable (and thus no hashbang).  The files exist to provide Python examples of how to write code that uses the lib, not to act as utilities (executable bins) that people should run to do things.  They're just example Python files, and, as such, I think should remain as is.
I'm going to close the PR, but please feel free to comment if you have a different view.  I'm open to being convinced otherwise.  And thanks for the contribution!

No problem, it is a respectable opinion :-) It is your project after all :-)


github: transfer govim repo to https://github.com/govim/govim?

I've created https://github.com/govim as a placeholder. Feels like this repository would be better housed there.
Not least because we can then do small things like preventing people pushing commits to master (which is not possible with personal repos like this)
Should be able to transfer the repository I guess?
Points to consider:

[ ] Will the transfer automatically put in place repository redirects?
[ ] Are there potentially any issues with the module changing path?

[ ] For cmd/govim
[ ] For the github.com/myitcv/govim package

...

cc @mvdan @leitzler @milesbxf for thoughts?

Sounds good to me, we need a logo! :)

Can you not accomplish the same with branch protection on your personal repo?
Up to you - I think this is personal preference. I generally only consider organizations when a project is either large or worked on actively by many people. It could also encourage others to take a more active role. So it seems fine to me :)

Can you not accomplish the same with branch protection on your personal repo?

Unfortunately not, just confirmed by GitHub support:

That option is only available for Organization owned repositories, I'm afraid. It can't be used on repository belonging to an individual user account.

where "that option" is a reference to the "Restrict who can push to this branch" option on https://help.github.com/en/articles/enabling-branch-restrictions.

I think this is personal preference

I'm not fussed either way. The transfer will be low cost now, higher cost in the future... doesn't seem to be any arguments against moving it... so let's go for it!

I've created https://github.com/govim as a placeholder. Feels like this repository would be better housed there.

I think it is better for community engagement too.

Points to consider:

[ ]  Will the transfer automatically put in place repository redirects?

When I transferred the rclone repo everything was transferred, code, issues, prs, wiki, project.
Github put in redirects from the old repo to the new repo which worked perfectly.
I had to press the button to do the transfer twice - the first did nothing so I contacted github support and they said to do it again!

[ ]  Are there potentially any issues with the module changing path?

[ ]  For cmd/govim
[ ]  For the github.com/myitcv/govim package

...

Fixing up the code with s/oldrepo/newrepo/ was easy enough and module and non modules builds carried on working.

https://github.com/rclone/rclone/commit/57d5de6fba12832469c27ee4eae6c2d4aa6a988f - fixing up the paths
https://github.com/rclone/rclone/commit/f347514f62cc7b9f373520b72b05cfdb1e7de2ae - fixing CI badges

The CI was a bit more annoying and probably took most of the time getting it working properly again.  Travis required a lot of reauthenticating but I had to make a whole new account for Appveyor.  I think Azure pipelines was OK too.
I wrote a doc for contributors on how to fix their dev environments which you are welcome to steal
Note in particular the git remote seturl which is much better than using deleting the remote and readding it as it doesn't reset all the branch upstreams.
https://github.com/rclone/rclone/wiki/What-to-do-after-the-repository-move

Thanks very much @ncw!

This is now complete. I updated the description of this issue with the steps followed.
Thanks very much @ncw for inspiring and supporting the change with your feedback!


Add an auto-restart function after the game is over

I think it would be a good idea to add a function so that after a game, it restarts automatically

wow absolutely revolutionary, hasnt been thought of before

code are in but is broken


business show = on the fright stage

With Uncle Dutch and Turk Young

Business show is now a synonym for fright stage and is also mentioned at the beginning/end.


Remove obsolete run deps

Not present anymore as of 1.11.1

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Indicators square dots

hello!
At some times, we need a RectanglePageIndicator, so could you can make it ?  thank you !

I believe i can :)

It will square dots bybyhe way :)

If my work is helping you, please help me back: https://xamarinhq.wufoo.com/forms/nominate-a-xamarin-mvp/

This is what i have done that is community visible:

Alex has been pretty active sharing his knowledge and answering questions in StackOverflow, GitHub and Xamarin Forums.
Alex has been open sourcing Xamarin components and plugin with the community, with more of 10 repositories being maintained on GitHub.
One of Alex's Xamarin.Forms plugin, the CarouselView, is currently being pulled into the Xamarin.Forms core (xamarin/Xamarin.Forms#853). As an stable widget, this control has become the standard for Xamarin.Forms carousel screens with more that 16K downloads in NuGet and more than 100 solved and not open issues.
Another Alex's plugin, the SegmentedControl (https://github.com/alexrainman/SegmentedControl), is a big candidate to be also pulled into Xamarin.Forms core.
Plus Xamarin.Forms UI related plugins, Alex shares plugins to set app icon Badge, Google Analytics and KeyChain access.
Alex shares his knowledge about cutting edge technologies like Couchbase NoSQL and MobileIron integration with Xamarin Apps.


Pick MSVC -arch: based on CARGO_CFG_TARGET_FEATURE for x86 / x86_64 targets

This is my first contribution to cc-rs, and I'm more than open to any critique or questions.

As it is now, -arch:IA32 is passed for 32-bit builds if the target is i586 or clang-cl is being used. This explicit override often breaks assumptions and always downgrades 32-bit clang-cl builds from the default -arch:SSE2 to -arch:IA32, potentially impacting performance.
This PR attempts to only downgrade when there's reason to, and otherwise attempt to pick a target -arch that matches the features set in CARGO_CFG_TARGET_FEATURE. Picking an -arch that follows CARGO_CFG_TARGET_FEATURE also means crates (both x86 and x86_64) that opt into support for AVX or AVX2 through target-feature will now also have any dependencies built with cc benefit from that.

I'm using clang-cl to build a few 32-bit projects, so I'm most likely running into issues with this far more often than most people are. While usually it's at most a bit annoying that cc builds aren't respecting target-feature, it actually breaks my builds.
Today I ran into this same issue again when trying to integrate mimalloc_rust, which requires SSE2 for atomics.
Since this is the third crate I've attempted to use that has issues due to the same forced override, I decided to try fix the problem in cc-rs instead of forking yet another crate just to add .flag_if_supported("-arch:SSE2") to be able to build it successfully.
I opted to not add support for AVX512 for now since MSVC bundles all AVX512 extensions under the same -arch:AVX512, while Rust treats each extension as a separate feature. Adding support for it is possible, but I don't need it myself and I don't know how to do it cleanly, so I'll leave that to someone else. For now it should fallback to AVX2, which is still an upgrade over the previous IA32 (clang-cl) / SSE2 (cl.exe).
It's also possible to do the same thing for ARM builds, but since I have no experience with ARM I'll leave that task too for someone else.

Hm, I was under the impression that I586 was intended explicitly for older CPUs that do not have newer instructions so is this not working as intended? Otherwise, how is it different from i686?

It is, but the forced downgrade on clang-cl is not currently checking for i586. It's unconditionally passing -arch:IA32 for any arch containing 86 that doesn't also contain x86_64.
For "normal" MSVC it's already working "as intended", it's just that "as intended" in this case means "assume the target supports SSE2 and nothing else" (MSVC's default). Which is completely fine, but means that even if your target supports AVX instructions MSVC will not use them.
I had issues due to the former part, and implemented the latter while I was at it since I was already working on making it slightly smarter.


README.md: Fix a result.

I correct a line in your README.md

Coverage remained the same at 100.0% when pulling e43fbc35b71e625a50c1835a54ee95d3ca39cd96 on Ashkanph:master into 763889f531112ed3c47b12e3bce70138c7f788b7 on scniro:master.

Coverage remained the same at 100.0% when pulling e43fbc35b71e625a50c1835a54ee95d3ca39cd96 on Ashkanph:master into 763889f531112ed3c47b12e3bce70138c7f788b7 on scniro:master.


Properties window cropped on scaled display

W10, fb2k 2.0 64-bit beta 13, OpenLyrics 1.3
Using a 4k display with font scaling 125%. The Properties window appears to have the contents scaled as expected, but as the Properties dialog is not resizeable, part of the contents is chopped off.
Steps to reproduce

Set Windows font scaling to 125% (or anything higher than 100%)
Open foobar Properties
Navigate to the OpenLyrics tab

Expected behavior
Either disable font scaling (as many other properties dialogs do) or make the dialog content scrollable.

Apparently any window created with CreateWindow needs to have it's items manually repositioned as documented by Microsoft, here.
On the other hand, apparently anything created with DialogBox* will be scaled automatically (for example see the wcap code here and here).


how to get the coordinate for a soft robot?

For a sphere:
robot_fem = scene.add_entity(
    morph=gs.morphs.Sphere(
        pos=(0.5, -0.2, 0.3),
        radius=0.1,
    ),
    material=gs.materials.FEM.Muscle(
        E=E,
        nu=nu,
        rho=rho,
        model='stable_neohooken',
    ),
)

It consist of many points and the sphere would elastic deformation. How can I get the coordinate for all points in the simulation?

@zswang666 can you take a look

hello ， I am also very concerned about this issue

state = robot_fem.get_state()
pos = state.pos # positions of all vertices in the elements
vel = state.vel # velocities of all vertices in the elements


integrations url processor doesn't implement support for private gitlab repos

Integrations for gitlab private repos do not work; only public repos can be pulled from, e.g. for catalog items.
Expected Behavior

The gitlab API (or different auth headers) need to be used with the token in order to get files from a private repo.
Current Behavior

Fails to load the referenced file.
Possible Solution

Use gitlab API endpoint to retrieve file content?
Steps to Reproduce

Make a private mirror of backstage/backstage.
Create an access token.
Modify example app-config.yaml (e.g. in app-config.local.yaml) to point a scaffolder entry at the private gitlab repo file. For example: the Documentation Template.
Run the demo server with yarn dev

[1] 2021-01-15T15:10:06.774Z catalog warn Failed item in location url:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, Error: YAML error, YAMLSemanticError: Implicit map keys need to be on a single line
[1]     at Module.generalError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:59:56)
[1]     at parseEntityYaml (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/util/parse.ts:45:66)
[1]     at parseEntityYaml.next (<anonymous>)
[1]     at UrlReaderProcessor.readLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/UrlReaderProcessor.ts:64:18)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:142:13)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin

Tracing the code shows https://github.com/backstage/backstage/blob/9d7b50f2f36f599df4c9ceacb18676fc0fdf72bc/packages/backend-common/src/reading/GitlabUrlReader.ts#L56 generates a GET with a PRIVATE-TOKEN header.
Gitlab only supports the PRIVATE-TOKEN header for the API, not file URLs.
Context

Trying to use backstage with private repos (on prem gitlab enterprise and/or github.com) and can't reference individual git-hosted files with the url processor.
The current workaround is to make the repos public.
Your Environment

NodeJS Version (v12): v12.20.0
Operating System and Version (e.g. Ubuntu 14.04): MacOS Catalina 10.15.7 (19H114)
Browser Information: N/A

@freben per our discord messages.
@Rugvip looks like you might want to look at this... Some notes I made:
Given this URL: https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/customize/doc-template.yaml

":id" can be integer project ID or URL-encoded org/repo: cuit-ent-arch%2Fbackstage

https://gitlab.cc.columbia.edu/api/v4/projects/cuit-ent-arch%2Fbackstage/repository/files/doc-template.yaml?ref=customize

response:

{
    "file_name": "doc-template.yaml",
    "file_path": "doc-template.yaml",
    "size": 658,
    "encoding": "base64",
    "content_sha256": "a93c6061001f2cc87185932f9fd91614acbe67d545270f95b148bf05e22c175e",
    "ref": "customize",
    "blob_id": "36404f088ca56dde04ea71bdcb73fc730607740c",
    "commit_id": "f2c6e35ac4d981788ab2ddd352ffd2dc1f1911b8",
    "last_commit_id": "f2c6e35ac4d981788ab2ddd352ffd2dc1f1911b8",
    "content": "YXBpVmVyc2lvbjogYmFja3N0YWdlLmlvL3YxYWxwaGExCmtpbmQ6IFRlbXBsYXRlCm1ldGFkYXRhOgogIG5hbWU6IGRvY3MtdGVtcGxhdGUKICB0aXRsZTogRG9jdW1lbnRhdGlvbiBUZW1wbGF0ZQogIGRlc2NyaXB0aW9uOiBDcmVhdGUgYSBuZXcgc3RhbmRhbG9uZSBkb2N1bWVudGF0aW9uIHByb2plY3QKICB0YWdzOgogICAgLSByZWNvbW1lbmRlZAogICAgLSB0ZWNoZG9jcwogICAgLSBta2RvY3MKc3BlYzoKICBvd25lcjogYmFja3N0YWdlL3RlY2hkb2NzLWNvcmUKICB0ZW1wbGF0ZXI6IGNvb2tpZWN1dHRlcgogIHR5cGU6IGRvY3VtZW50YXRpb24KICBwYXRoOiAnLicKCiAgc2NoZW1hOgogICAgcmVxdWlyZWQ6CiAgICAgIC0gY29tcG9uZW50X2lkCiAgICAgIC0gZGVzY3JpcHRpb24KICAgIHByb3BlcnRpZXM6CiAgICAgIGNvbXBvbmVudF9pZDoKICAgICAgICB0aXRsZTogTmFtZQogICAgICAgIHR5cGU6IHN0cmluZwogICAgICAgIGRlc2NyaXB0aW9uOiBVbmlxdWUgbmFtZSBvZiB0aGUgY29tcG9uZW50CiAgICAgIGRlc2NyaXB0aW9uOgogICAgICAgIHRpdGxlOiBEZXNjcmlwdGlvbgogICAgICAgIHR5cGU6IHN0cmluZwogICAgICAgIGRlc2NyaXB0aW9uOiBIZWxwIG90aGVycyB1bmRlcnN0YW5kIHdoYXQgdGhlc2UgZG9jcyBhcmUgYWJvdXQuCg=="
}

@freben per our discord messages.
@Rugvip looks like you might want to look at this... Some notes I made:
Given this URL: https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/customize/doc-template.yaml

":id" can be integer project ID or URL-encoded org/repo: cuit-ent-arch%2Fbackstage

https://gitlab.cc.columbia.edu/api/v4/projects/cuit-ent-arch%2Fbackstage/repository/files/doc-template.yaml?ref=customize

response:

{
    "file_name": "doc-template.yaml",
    "file_path": "doc-template.yaml",
    "size": 658,
    "encoding": "base64",
    "content_sha256": "a93c6061001f2cc87185932f9fd91614acbe67d545270f95b148bf05e22c175e",
    "ref": "customize",
    "blob_id": "36404f088ca56dde04ea71bdcb73fc730607740c",
    "commit_id": "f2c6e35ac4d981788ab2ddd352ffd2dc1f1911b8",
    "last_commit_id": "f2c6e35ac4d981788ab2ddd352ffd2dc1f1911b8",
    "content": "YXBpVmVyc2lvbjogYmFja3N0YWdlLmlvL3YxYWxwaGExCmtpbmQ6IFRlbXBsYXRlCm1ldGFkYXRhOgogIG5hbWU6IGRvY3MtdGVtcGxhdGUKICB0aXRsZTogRG9jdW1lbnRhdGlvbiBUZW1wbGF0ZQogIGRlc2NyaXB0aW9uOiBDcmVhdGUgYSBuZXcgc3RhbmRhbG9uZSBkb2N1bWVudGF0aW9uIHByb2plY3QKICB0YWdzOgogICAgLSByZWNvbW1lbmRlZAogICAgLSB0ZWNoZG9jcwogICAgLSBta2RvY3MKc3BlYzoKICBvd25lcjogYmFja3N0YWdlL3RlY2hkb2NzLWNvcmUKICB0ZW1wbGF0ZXI6IGNvb2tpZWN1dHRlcgogIHR5cGU6IGRvY3VtZW50YXRpb24KICBwYXRoOiAnLicKCiAgc2NoZW1hOgogICAgcmVxdWlyZWQ6CiAgICAgIC0gY29tcG9uZW50X2lkCiAgICAgIC0gZGVzY3JpcHRpb24KICAgIHByb3BlcnRpZXM6CiAgICAgIGNvbXBvbmVudF9pZDoKICAgICAgICB0aXRsZTogTmFtZQogICAgICAgIHR5cGU6IHN0cmluZwogICAgICAgIGRlc2NyaXB0aW9uOiBVbmlxdWUgbmFtZSBvZiB0aGUgY29tcG9uZW50CiAgICAgIGRlc2NyaXB0aW9uOgogICAgICAgIHRpdGxlOiBEZXNjcmlwdGlvbgogICAgICAgIHR5cGU6IHN0cmluZwogICAgICAgIGRlc2NyaXB0aW9uOiBIZWxwIG90aGVycyB1bmRlcnN0YW5kIHdoYXQgdGhlc2UgZG9jcyBhcmUgYWJvdXQuCg=="
}

Thanks!  Double check that it's the one that actually gets used - i.e., if it's overridden in app-config.local.yaml, then post that one, of course with the token censored.

Thanks!  Double check that it's the one that actually gets used - i.e., if it's overridden in app-config.local.yaml, then post that one, of course with the token censored.

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is here. The gitlab integration works for our private repos for catalog files as well as techdocs, etc.

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is here. The gitlab integration works for our private repos for catalog files as well as techdocs, etc.

Thanks!  Double check that it's the one that actually gets used - i.e., if it's overridden in app-config.local.yaml, then post that one, of course with the token censored.

integrations:
  github:
    - host: github.com
      token:
        $env: GITHUB_TOKEN
  gitlab:
    - host: gitlab.cc.columbia.edu
      token:
        $env: GITLAB_TOKEN
      apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4

Thanks!  Double check that it's the one that actually gets used - i.e., if it's overridden in app-config.local.yaml, then post that one, of course with the token censored.

integrations:
  github:
    - host: github.com
      token:
        $env: GITHUB_TOKEN
  gitlab:
    - host: gitlab.cc.columbia.edu
      token:
        $env: GITLAB_TOKEN
      apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is here. The gitlab integration works for our private repos for catalog files as well as techdocs, etc.

My understanding is these types are being deprecated and replaced by integrations type: url. I guess I will also have to add a gitlab processor to make this error go away:
[1] 2021-01-15T16:04:43.214Z catalog warn No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml type=plugin
[1] 2021-01-15T16:04:49.714Z catalog warn Failed item in location gitlab:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, InputError: No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml
[1]     at Module.inputError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:51:12)
[1]     at LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:159:72)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is here. The gitlab integration works for our private repos for catalog files as well as techdocs, etc.

My understanding is these types are being deprecated and replaced by integrations type: url. I guess I will also have to add a gitlab processor to make this error go away:
[1] 2021-01-15T16:04:43.214Z catalog warn No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml type=plugin
[1] 2021-01-15T16:04:49.714Z catalog warn Failed item in location gitlab:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, InputError: No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml
[1]     at Module.inputError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:51:12)
[1]     at LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:159:72)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is [here]

BTW, there's no need to get the project ID as it can also be a url-encoded org/repo

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is [here]

BTW, there's no need to get the project ID as it can also be a url-encoded org/repo

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is here. The gitlab integration works for our private repos for catalog files as well as techdocs, etc.

My understanding is these types are being deprecated and replaced by integrations type: url. I guess I will also have to add a gitlab processor to make this error go away:
[1] 2021-01-15T16:04:43.214Z catalog warn No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml type=plugin
[1] 2021-01-15T16:04:49.714Z catalog warn Failed item in location gitlab:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, InputError: No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml
[1]     at Module.inputError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:51:12)
[1]     at LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:159:72)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin

I explicitly added the processor but still seeing the same error:
diff --git a/app-config.yaml b/app-config.yaml
index 5866354a8..8653eb9cf 100644
--- a/app-config.yaml
+++ b/app-config.yaml
@@ -139,6 +139,12 @@ catalog:
         - Location

   processors:
+    gitlab:
+      providers:
+        - target: https://gitlab.cc.columbia.edu
+          token:
+            $env: GITLAB_TOKEN
+          apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4

@n2ygk if you try type: gitlab I believe this will work. The code for figuring out project ID/talking to the gitlab API is here. The gitlab integration works for our private repos for catalog files as well as techdocs, etc.

My understanding is these types are being deprecated and replaced by integrations type: url. I guess I will also have to add a gitlab processor to make this error go away:
[1] 2021-01-15T16:04:43.214Z catalog warn No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml type=plugin
[1] 2021-01-15T16:04:49.714Z catalog warn Failed item in location gitlab:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, InputError: No processor was able to read location gitlab https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml
[1]     at Module.inputError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:51:12)
[1]     at LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:159:72)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin

I explicitly added the processor but still seeing the same error:
diff --git a/app-config.yaml b/app-config.yaml
index 5866354a8..8653eb9cf 100644
--- a/app-config.yaml
+++ b/app-config.yaml
@@ -139,6 +139,12 @@ catalog:
         - Location

   processors:
+    gitlab:
+      providers:
+        - target: https://gitlab.cc.columbia.edu
+          token:
+            $env: GITLAB_TOKEN
+          apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4

That last config diff does not show a host field. Just double checking

That last config diff does not show a host field. Just double checking

That last config diff does not show a host field. Just double checking

Isn't the host inferred from the target? Anyway, I've added it and also did some experiments with gitlab vs url type.
Here's my config.app diffs from 9420aef1ae9409923e4dfa7bba9deb619e4bfe73:
backstage$ ls -l app-config.local.yaml
ls: app-config.local.yaml: No such file or directory
backstage$ git diff master app-config.yaml
diff --git a/app-config.yaml b/app-config.yaml
index 5866354a8..2af04d06f 100644
--- a/app-config.yaml
+++ b/app-config.yaml
@@ -110,10 +110,15 @@ integrations:
     #   rawBaseUrl: https://ghe.example.net/raw
     #   token:
     #     $env: GHE_TOKEN
+  # gitlab:
+  #   - host: gitlab.com
+  #     token:
+  #       $env: GITLAB_TOKEN
   gitlab:
-    - host: gitlab.com
+    - host: gitlab.cc.columbia.edu
       token:
         $env: GITLAB_TOKEN
+      apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4
   bitbucket:
     - host: bitbucket.org
       username:
@@ -149,6 +154,13 @@ catalog:
         #   apiBaseUrl: https://ghe.example.net/api
         #   token:
         #     $env: GHE_TOKEN
+    # gitlab:
+    #   providers:
+    #     - host: gitlab.cc.columbia.edu
+    #       target: https://gitlab.cc.columbia.edu
+    #       token:
+    #         $env: GITLAB_TOKEN
+    #       apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4
     ldapOrg:
       ### Example for how to add your enterprise LDAP server
       # providers:
@@ -187,9 +199,6 @@ catalog:
     # Example component for github-actions
     - type: url
       target: https://github.com/backstage/backstage/blob/master/plugins/github-actions/examples/sample.yaml
-    # Example component for TechDocs
-    - type: url
-      target: https://github.com/backstage/backstage/blob/master/plugins/techdocs-backend/examples/documented-component/catalog-info.yaml
     # Backstage example APIs
     - type: url
       target: https://github.com/backstage/backstage/blob/master/packages/catalog-model/examples/all-apis.yaml
@@ -203,8 +212,16 @@ catalog:
     - type: url
       target: https://github.com/backstage/backstage/blob/master/packages/catalog-model/examples/all-domains.yaml
     # Backstage example templates
+    # - type: url
+    #   target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/all-templates.yaml
+    ### try a few here:
     - type: url
-      target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/all-templates.yaml
+      target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/react-ssr-template/template.yaml
+    # Example component for TechDocs
+    - type: url
+      target: https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml
+    #   target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/docs-template/template.yaml
+
     # Backstage example groups and users
     - type: url
       target: https://github.com/backstage/backstage/blob/master/packages/catalog-model/examples/acme-corp.yaml

(I commented out the all-templates to make things more clear and configured the react-ssr-template and the gitlab-hosted TechDocs template)
Here's the log, showing the error to load the gitlab template:
yarn run v1.22.1
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace example-app start
$ yarn workspace example-backend start
$ backstage-cli app:serve
$ backstage-cli backend:dev
[0] Loaded config from app-config.yaml
[0] ℹ ｢wds｣: Project is running at http://localhost:3000/
[0] ℹ ｢wds｣: webpack output is served from /
[0] ℹ ｢wds｣: Content not from webpack is served from /Users/alan/src/backstage/packages/app/public
[0] ℹ ｢wds｣: 404s will fallback to /index.html
[1] Build succeeded
[0] ℹ ｢wdm｣: wait until bundle finished: /create
[1] 2021-01-15T18:24:04.404Z backstage info Loaded config from app-config.yaml
[1] 2021-01-15T18:24:04.409Z backstage info Created UrlReader predicateMux{readers=azure{host=dev.azure.com,authed=false},bitbucket{host=bitbucket.org,authed=false},github{host=github.com,authed=true},gitlab{host=gitlab.cc.columbia.edu,authed=true},gitlab{host=gitlab.com,authed=false},fallback=fetch{}}
[1] 2021-01-15T18:24:04.522Z catalog info Locations Refresh: Beginning locations refresh type=plugin component=catalog-all-locations-refresh
[1] 2021-01-15T18:24:04.539Z scaffolder warn Skipping azure scaffolding provider, Missing required config value at 'scaffolder.azure.api.token' type=plugin
[1] 2021-01-15T18:24:04.539Z scaffolder warn Skipping bitbucket scaffolding provider, Missing required config value at 'scaffolder.bitbucket.api.username' type=plugin
[1] 2021-01-15T18:24:04.544Z catalog info Locations Refresh: Visiting 1 locations type=plugin component=catalog-all-locations-refresh
[1] 2021-01-15T18:24:04.544Z catalog info Locations Refresh: Refreshing location bootstrap:bootstrap type=plugin component=catalog-all-locations-refresh
[1] 2021-01-15T18:24:04.582Z auth info Configuring provider, google type=plugin
[1] 2021-01-15T18:24:04.583Z auth warn Skipping google auth provider, Missing required config value at 'auth.providers.google.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.583Z auth info Configuring provider, github type=plugin
[1] 2021-01-15T18:24:04.584Z auth info Configuring provider, gitlab type=plugin
[1] 2021-01-15T18:24:04.586Z auth info Configuring provider, saml type=plugin
[1] 2021-01-15T18:24:04.587Z auth info Configuring provider, okta type=plugin
[1] 2021-01-15T18:24:04.587Z auth warn Skipping okta auth provider, Missing required config value at 'auth.providers.okta.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.587Z auth info Configuring provider, oauth2 type=plugin
[1] 2021-01-15T18:24:04.588Z auth warn Skipping oauth2 auth provider, Missing required config value at 'auth.providers.oauth2.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.588Z auth info Configuring provider, oidc type=plugin
[1] 2021-01-15T18:24:04.588Z auth warn Skipping oidc auth provider, Missing required config value at 'auth.providers.oidc.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.588Z auth info Configuring provider, auth0 type=plugin
[1] 2021-01-15T18:24:04.588Z auth warn Skipping auth0 auth provider, Missing required config value at 'auth.providers.auth0.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.589Z auth info Configuring provider, microsoft type=plugin
[1] 2021-01-15T18:24:04.589Z auth warn Skipping microsoft auth provider, Missing required config value at 'auth.providers.microsoft.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.589Z auth info Configuring provider, onelogin type=plugin
[1] 2021-01-15T18:24:04.589Z auth warn Skipping onelogin auth provider, Missing required config value at 'auth.providers.onelogin.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.591Z techdocs info Creating Local publisher for TechDocs type=plugin
[1] 2021-01-15T18:24:04.593Z kubernetes info Initializing Kubernetes backend type=plugin
[1] 2021-01-15T18:24:04.597Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://circleci.com/api/v1.1 type=plugin
[1] 2021-01-15T18:24:04.597Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/circleci/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.599Z proxy info [HPM] Proxy created: [Function: filter]  ->  http://localhost:8080 type=plugin
[1] 2021-01-15T18:24:04.599Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/jenkins/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.599Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.travis-ci.com type=plugin
[1] 2021-01-15T18:24:04.606Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/travisci/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.606Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.newrelic.com/v2 type=plugin
[1] 2021-01-15T18:24:04.606Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/newrelic/apm/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.pagerduty.com type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/pagerduty/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.buildkite.com/v2/ type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/buildkite/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.608Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://sentry.io/api/ type=plugin
[1] 2021-01-15T18:24:04.608Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/sentry/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.670Z app info Serving static app content from /Users/alan/src/backstage/packages/app/dist type=plugin
[1] 2021-01-15T18:24:04.740Z app info Replacing injected env config in main.7d404d73.chunk.js type=plugin
[1] 2021-01-15T18:24:04.748Z backstage info Initializing http server
[1] 2021-01-15T18:24:04.751Z backstage info Listening on :7000
[1] 2021-01-15T18:24:06.910Z backstage info ::1 - - [15/Jan/2021:18:24:06 +0000] "GET /api/catalog/entities?filter=kind=Template HTTP/1.1" 200 2 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" type=incomingRequest
[1] 2021-01-15T18:24:11.697Z catalog warn Failed item in location url:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, Error: YAML error, YAMLSemanticError: Implicit map keys need to be on a single line
[1]     at Module.generalError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:59:56)
[1]     at parseEntityYaml (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/util/parse.ts:45:66)
[1]     at parseEntityYaml.next (<anonymous>)
[1]     at UrlReaderProcessor.readLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/UrlReaderProcessor.ts:64:18)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:142:13)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin
[1] 2021-01-15T18:24:11.697Z catalog info Read 57 entities from location bootstrap:bootstrap in 7.2s type=plugin
[1] 2021-01-15T18:24:11.791Z catalog info Wrote 57 entities from location bootstrap:bootstrap in 94ms type=plugin
[1] 2021-01-15T18:24:11.792Z catalog info Locations Refresh: Completed locations refresh in 7.3s type=plugin component=catalog-all-locations-refresh
[0] ℹ ｢wdm｣: Compiled successfully.
[1] 2021-01-15T18:24:29.855Z backstage info ::1 - - [15/Jan/2021:18:24:29 +0000] "GET /api/catalog/entities?filter=kind=Template HTTP/1.1" 200 923 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" type=incomingRequest

And a screenshot of localhost:3000/create:

All that above is with a private gitlab repo:

Now make the same repo public:

and voila, no errors logged and this:

I've traced this code, pulled out the URLs and headers and tested with Postman and it's pretty clear that the code is setting a private-token header on the URL as given whereas one I believe has to use the API endpoint to actually get the file content as a json document. When the repo is public, it "just works."

That last config diff does not show a host field. Just double checking

Isn't the host inferred from the target? Anyway, I've added it and also did some experiments with gitlab vs url type.
Here's my config.app diffs from 9420aef1ae9409923e4dfa7bba9deb619e4bfe73:
backstage$ ls -l app-config.local.yaml
ls: app-config.local.yaml: No such file or directory
backstage$ git diff master app-config.yaml
diff --git a/app-config.yaml b/app-config.yaml
index 5866354a8..2af04d06f 100644
--- a/app-config.yaml
+++ b/app-config.yaml
@@ -110,10 +110,15 @@ integrations:
     #   rawBaseUrl: https://ghe.example.net/raw
     #   token:
     #     $env: GHE_TOKEN
+  # gitlab:
+  #   - host: gitlab.com
+  #     token:
+  #       $env: GITLAB_TOKEN
   gitlab:
-    - host: gitlab.com
+    - host: gitlab.cc.columbia.edu
       token:
         $env: GITLAB_TOKEN
+      apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4
   bitbucket:
     - host: bitbucket.org
       username:
@@ -149,6 +154,13 @@ catalog:
         #   apiBaseUrl: https://ghe.example.net/api
         #   token:
         #     $env: GHE_TOKEN
+    # gitlab:
+    #   providers:
+    #     - host: gitlab.cc.columbia.edu
+    #       target: https://gitlab.cc.columbia.edu
+    #       token:
+    #         $env: GITLAB_TOKEN
+    #       apiBaseUrl: https://gitlab.cc.columbia.edu/api/v4
     ldapOrg:
       ### Example for how to add your enterprise LDAP server
       # providers:
@@ -187,9 +199,6 @@ catalog:
     # Example component for github-actions
     - type: url
       target: https://github.com/backstage/backstage/blob/master/plugins/github-actions/examples/sample.yaml
-    # Example component for TechDocs
-    - type: url
-      target: https://github.com/backstage/backstage/blob/master/plugins/techdocs-backend/examples/documented-component/catalog-info.yaml
     # Backstage example APIs
     - type: url
       target: https://github.com/backstage/backstage/blob/master/packages/catalog-model/examples/all-apis.yaml
@@ -203,8 +212,16 @@ catalog:
     - type: url
       target: https://github.com/backstage/backstage/blob/master/packages/catalog-model/examples/all-domains.yaml
     # Backstage example templates
+    # - type: url
+    #   target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/all-templates.yaml
+    ### try a few here:
     - type: url
-      target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/all-templates.yaml
+      target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/react-ssr-template/template.yaml
+    # Example component for TechDocs
+    - type: url
+      target: https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml
+    #   target: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/docs-template/template.yaml
+
     # Backstage example groups and users
     - type: url
       target: https://github.com/backstage/backstage/blob/master/packages/catalog-model/examples/acme-corp.yaml

(I commented out the all-templates to make things more clear and configured the react-ssr-template and the gitlab-hosted TechDocs template)
Here's the log, showing the error to load the gitlab template:
yarn run v1.22.1
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace example-app start
$ yarn workspace example-backend start
$ backstage-cli app:serve
$ backstage-cli backend:dev
[0] Loaded config from app-config.yaml
[0] ℹ ｢wds｣: Project is running at http://localhost:3000/
[0] ℹ ｢wds｣: webpack output is served from /
[0] ℹ ｢wds｣: Content not from webpack is served from /Users/alan/src/backstage/packages/app/public
[0] ℹ ｢wds｣: 404s will fallback to /index.html
[1] Build succeeded
[0] ℹ ｢wdm｣: wait until bundle finished: /create
[1] 2021-01-15T18:24:04.404Z backstage info Loaded config from app-config.yaml
[1] 2021-01-15T18:24:04.409Z backstage info Created UrlReader predicateMux{readers=azure{host=dev.azure.com,authed=false},bitbucket{host=bitbucket.org,authed=false},github{host=github.com,authed=true},gitlab{host=gitlab.cc.columbia.edu,authed=true},gitlab{host=gitlab.com,authed=false},fallback=fetch{}}
[1] 2021-01-15T18:24:04.522Z catalog info Locations Refresh: Beginning locations refresh type=plugin component=catalog-all-locations-refresh
[1] 2021-01-15T18:24:04.539Z scaffolder warn Skipping azure scaffolding provider, Missing required config value at 'scaffolder.azure.api.token' type=plugin
[1] 2021-01-15T18:24:04.539Z scaffolder warn Skipping bitbucket scaffolding provider, Missing required config value at 'scaffolder.bitbucket.api.username' type=plugin
[1] 2021-01-15T18:24:04.544Z catalog info Locations Refresh: Visiting 1 locations type=plugin component=catalog-all-locations-refresh
[1] 2021-01-15T18:24:04.544Z catalog info Locations Refresh: Refreshing location bootstrap:bootstrap type=plugin component=catalog-all-locations-refresh
[1] 2021-01-15T18:24:04.582Z auth info Configuring provider, google type=plugin
[1] 2021-01-15T18:24:04.583Z auth warn Skipping google auth provider, Missing required config value at 'auth.providers.google.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.583Z auth info Configuring provider, github type=plugin
[1] 2021-01-15T18:24:04.584Z auth info Configuring provider, gitlab type=plugin
[1] 2021-01-15T18:24:04.586Z auth info Configuring provider, saml type=plugin
[1] 2021-01-15T18:24:04.587Z auth info Configuring provider, okta type=plugin
[1] 2021-01-15T18:24:04.587Z auth warn Skipping okta auth provider, Missing required config value at 'auth.providers.okta.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.587Z auth info Configuring provider, oauth2 type=plugin
[1] 2021-01-15T18:24:04.588Z auth warn Skipping oauth2 auth provider, Missing required config value at 'auth.providers.oauth2.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.588Z auth info Configuring provider, oidc type=plugin
[1] 2021-01-15T18:24:04.588Z auth warn Skipping oidc auth provider, Missing required config value at 'auth.providers.oidc.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.588Z auth info Configuring provider, auth0 type=plugin
[1] 2021-01-15T18:24:04.588Z auth warn Skipping auth0 auth provider, Missing required config value at 'auth.providers.auth0.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.589Z auth info Configuring provider, microsoft type=plugin
[1] 2021-01-15T18:24:04.589Z auth warn Skipping microsoft auth provider, Missing required config value at 'auth.providers.microsoft.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.589Z auth info Configuring provider, onelogin type=plugin
[1] 2021-01-15T18:24:04.589Z auth warn Skipping onelogin auth provider, Missing required config value at 'auth.providers.onelogin.development.clientId' type=plugin
[1] 2021-01-15T18:24:04.591Z techdocs info Creating Local publisher for TechDocs type=plugin
[1] 2021-01-15T18:24:04.593Z kubernetes info Initializing Kubernetes backend type=plugin
[1] 2021-01-15T18:24:04.597Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://circleci.com/api/v1.1 type=plugin
[1] 2021-01-15T18:24:04.597Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/circleci/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.599Z proxy info [HPM] Proxy created: [Function: filter]  ->  http://localhost:8080 type=plugin
[1] 2021-01-15T18:24:04.599Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/jenkins/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.599Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.travis-ci.com type=plugin
[1] 2021-01-15T18:24:04.606Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/travisci/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.606Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.newrelic.com/v2 type=plugin
[1] 2021-01-15T18:24:04.606Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/newrelic/apm/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.pagerduty.com type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/pagerduty/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://api.buildkite.com/v2/ type=plugin
[1] 2021-01-15T18:24:04.607Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/buildkite/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.608Z proxy info [HPM] Proxy created: [Function: filter]  ->  https://sentry.io/api/ type=plugin
[1] 2021-01-15T18:24:04.608Z proxy info [HPM] Proxy rewrite rule created: "^/api/proxy/sentry/api/" ~> "/" type=plugin
[1] 2021-01-15T18:24:04.670Z app info Serving static app content from /Users/alan/src/backstage/packages/app/dist type=plugin
[1] 2021-01-15T18:24:04.740Z app info Replacing injected env config in main.7d404d73.chunk.js type=plugin
[1] 2021-01-15T18:24:04.748Z backstage info Initializing http server
[1] 2021-01-15T18:24:04.751Z backstage info Listening on :7000
[1] 2021-01-15T18:24:06.910Z backstage info ::1 - - [15/Jan/2021:18:24:06 +0000] "GET /api/catalog/entities?filter=kind=Template HTTP/1.1" 200 2 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" type=incomingRequest
[1] 2021-01-15T18:24:11.697Z catalog warn Failed item in location url:https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/blob/master/templates/docs-template/template.yaml, Error: YAML error, YAMLSemanticError: Implicit map keys need to be on a single line
[1]     at Module.generalError (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/results.ts:59:56)
[1]     at parseEntityYaml (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/util/parse.ts:45:66)
[1]     at parseEntityYaml.next (<anonymous>)
[1]     at UrlReaderProcessor.readLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/processors/UrlReaderProcessor.ts:64:18)
[1]     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[1]     at async LocationReaders.handleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:142:13)
[1]     at async LocationReaders.read (webpack-internal:///../../plugins/catalog-backend/src/ingestion/LocationReaders.ts:74:11)
[1]     at async HigherOrderOperations.refreshSingleLocation (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:159:26)
[1]     at async HigherOrderOperations.refreshAllLocations (webpack-internal:///../../plugins/catalog-backend/src/ingestion/HigherOrderOperations.ts:138:9)
[1]     at async startRefresh (webpack-internal:///../../plugins/catalog-backend/src/util/runPeriodically.ts:42:9) type=plugin
[1] 2021-01-15T18:24:11.697Z catalog info Read 57 entities from location bootstrap:bootstrap in 7.2s type=plugin
[1] 2021-01-15T18:24:11.791Z catalog info Wrote 57 entities from location bootstrap:bootstrap in 94ms type=plugin
[1] 2021-01-15T18:24:11.792Z catalog info Locations Refresh: Completed locations refresh in 7.3s type=plugin component=catalog-all-locations-refresh
[0] ℹ ｢wdm｣: Compiled successfully.
[1] 2021-01-15T18:24:29.855Z backstage info ::1 - - [15/Jan/2021:18:24:29 +0000] "GET /api/catalog/entities?filter=kind=Template HTTP/1.1" 200 923 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" type=incomingRequest

And a screenshot of localhost:3000/create:

All that above is with a private gitlab repo:

Now make the same repo public:

and voila, no errors logged and this:

I've traced this code, pulled out the URLs and headers and tested with Postman and it's pretty clear that the code is setting a private-token header on the URL as given whereas one I believe has to use the API endpoint to actually get the file content as a json document. When the repo is public, it "just works."

I have the same (?) or similar problem. Getting the catalog-info.yaml is possible in a public repo on our self-hosted gitlab, but not in a private repo (I've tested the token).
The error I get: "YAML error, YAMLSemanticError: Implicit map keys need to be on a single line"
I added this to the backstage config:
integrations:
  gitlab:
    - host: gitlab.wklive.net
      apiBaseUrl: https://gitlab.wklive.net/api/v4
      [token]

I have the same (?) or similar problem. Getting the catalog-info.yaml is possible in a public repo on our self-hosted gitlab, but not in a private repo (I've tested the token).
The error I get: "YAML error, YAMLSemanticError: Implicit map keys need to be on a single line"
I added this to the backstage config:
integrations:
  gitlab:
    - host: gitlab.wklive.net
      apiBaseUrl: https://gitlab.wklive.net/api/v4
      [token]

That specific error almost always means that the response was HTML, not yaml. So, most likely a login page or similar.

That specific error almost always means that the response was HTML, not yaml. So, most likely a login page or similar.

@n2ygk all you have to do is to add a -/ in the URL xD, like this:
https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/-/blob/master/templates/docs-template/template.yaml
This is because of the logic in the getGitLabFileFetchUrl function, if the URL contains -/ it will use the API, otherwise, it will do a simple fetch.
https://github.com/backstage/backstage/blob/master/packages/integration/src/gitlab/core.ts#L41
It could be rewritten following the same logic as the github integration, what do you think @freben?.

@n2ygk all you have to do is to add a -/ in the URL xD, like this:
https://gitlab.cc.columbia.edu/cuit-ent-arch/backstage/-/blob/master/templates/docs-template/template.yaml
This is because of the logic in the getGitLabFileFetchUrl function, if the URL contains -/ it will use the API, otherwise, it will do a simple fetch.
https://github.com/backstage/backstage/blob/master/packages/integration/src/gitlab/core.ts#L41
It could be rewritten following the same logic as the github integration, what do you think @freben?.

Huh. scoped URLs. Go figure... Didn't even know that was a thing. Aha! I see that our gitlab instance is on an old version (11.11.0) while newer versions will remove the legacy (no-dash) URLs so that will be a non-issue going forward.
Thanks @ruloweb!

Huh. scoped URLs. Go figure... Didn't even know that was a thing. Aha! I see that our gitlab instance is on an old version (11.11.0) while newer versions will remove the legacy (no-dash) URLs so that will be a non-issue going forward.
Thanks @ruloweb!

So how do you use private gitlab repos?   I have tired encoding the URL, providing the project id, changing ~/ to -/ and although I can curl the file no problem I am unable to register an existing component using the UI.


Lazy loading i18n files can cause issue with karma tests

Note
I may have misunderstood a few things, feel free to roast me if I'm mistaken. From using the lazy loading in a few places, I think I understood that the "lazy loaded" part gets merged into the "main" dictionary.

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report
[ ] Performance issue
[x] Feature request
[x] Documentation issue or request
[ ] Support request
[ ] Other... Please describe:

Current behavior
If you use the i18n lazy loading like in the documentation, for example :
providers: [{ provide: TRANSLOCO_SCOPE, useValue: 'authentication' }],
Then you can use *transloco="let t; read: 'authentication'" in a component and translate the lazy loaded part of the dictionary fine.
But then you hit a wall in the karma test, because if you import the /authentication/en.json and give it to the TranslocoTestingModule, then the useValue: 'authentication' will fail and your unit test crashes. (Due to the fact that you gave the lazy loaded part, so useValue will try to start from a key that potentially doesn't exist).
If you import the base en.json, same will happen as the lazy loaded part haven't been merged. I'm currently working around this issue by manually merging the two json files together to make my tests work, but it's not very elegant.
import en from '../../../../../assets/i18n/en.json';
import en_authentication from '../../../../../assets/i18n/authentication/en.json';
...
 let merged = Object.assign({}, en);
  merged = Object.assign(merged, { authentication: en_authentication });
...
TranslocoTestingModule.withLangs(
          { en: merged }, { defaultLang: 'en' }
        )

Expected behavior
I don't know what's the prettiest way to handle this, maybe a config that we can give to TranslocoTestingModule to say "yo, ignore the useValue that you may encounter because I am providing a lazy loaded part of the i18n dictionary" ?
Or maybe a way to provide an array of json files that will internally get merged together ?
TranslocoTestingModule.withLangs(
          { en: TranslocoTestingModule.merge(en, en_lazy_partial) }, { defaultLang: 'en' }
        )

Minimal reproduction of the problem with instructions

Base i18n/en.json
Lazy loaded i18n/directory/en.json
A lazy loaded module with a component inside that uses i18n/directory/en.json with useValue
A karma unit test that tries to create the component

What is the motivation / use case for changing the behavior?
Feels tedious to manually merge the json files for every unit tests that will potentially have this issue (any lazy loaded i18n dictionaries).
Environment

Angular version: 8.2.4
Transloco: 1.7.9

Hi @Beerbossa.
We currently aren't going to add an API for merging scopes for testing.
You could provide a mock that includes the scope languages or merge them as you suggested.
Having said that, we might add an API for it in the future.

@Beerbossa you should pass the scope with the lang:
TranslocoTestingModule.withLangs(
  {
	en,
    'authentication/en': authentication
  }
)

In V2 we'll take care of the lang postfix so you'll be able to do:
TranslocoTestingModule.withLangs(
  {
	en,
    authentication
  }
)

@Beerbossa you should pass the scope with the lang:
TranslocoTestingModule.withLangs(
  {
	en,
    'authentication/en': authentication
  }
)

In V2 we'll take care of the lang postfix so you'll be able to do:
TranslocoTestingModule.withLangs(
  {
	en,
    authentication
  }
)

Thank you, I knew I was missing some kind of way to "merge" them in a cleaner way, that's good for me ! Appreciate the tip and the plan for V2 !

@NetanelBasal
Actually quick question, it seems to fail karma tests when I do it this way :

Am I missing something ? Sorry to bother you again

@Beerbossa could you please write here the HTML template of your component?

@Beerbossa could you please write here the HTML template of your component?

A lot have changed since then and everything works well merging the json files using the edit of my last comment, but basically it looked like that :
lazyLoaded.module.ts <---- This would lazy load the i18n 'authentication/en.json'
@NgModule({
  declarations: [
    DummyComponent
  ],
  providers: [
    { provide: TRANSLOCO_SCOPE, useValue: 'authentication' },
  ],
  imports: [
    CommonModule,
    SharedModule,
  ],
})

dummy-component.html <--- This would create the variable "t" to reference "t.authentication"
<div *transloco="let t; read: 'authentication' " class="main-container">
  <span>{{ t.login_title }}</span>
</div>

dummy-component.spec.ts <------ This would not merge "i18n/authentication/en.json" with "i18n/en.json".
import enfrom '../../../../../assets/i18n/en.json';
import authentication from '../../../../../assets/i18n/authentication/en.json';
TranslocoTestingModule.withLangs(
  {
	en,
    'authentication/en': authentication
  }
)

@Beerbossa I have updated the playground lazy page with your case, and it seems to be working fine at version 2.
see: https://github.com/ngneat/transloco/blob/master/src/app/lazy/lazy.component.spec.ts#L38
I would suggest you upgrade your code to use the new version and update your test as @NetanelBasal suggested.


How to stop a timer cron job based on uwsgi cron time signal

I tried the spooler task which works well.  And for the timer, cron tasks they also works well.  But I didn't find any API could help to stop/ cancel timer cron taskss at running time.
Is there a way to help do that?

Hi! I'm not sure that it's possible to stop timers/crons correctly, since UWSGI Python API does not support "unregister signal for all workers" feature (see). So, it's better to forward your question to uwsgi creators: unbit/uwsgi/issues.
However I think it's possible to re-register signals at least for lazy timers and crons thus disable execution of attached handlers, e.g.:
@cron_lazy(minute=-2, target='mule')
def print_every_2_minutes_after_call(signal_number):
     print('Cron task:', signal_number)

def enable_cron():
    print_every_2_minutes_after_call()

def disable_cron():
    task = print_every_2_minutes_after_call.uwsgi_task
    import uwsgi
    # the code below WON'T stop the cron, but remove handler so nothing is executed when
    # cron job is triggered
    uwsgi.register_signal(task.signal_id, task.target, None)

The code above should work when both enable_cron and disable_cron are executed in the same worker, so the usage is pretty limited.

Thanks for the reply.
One more question about how to make the uwsgi tasks visible? Like to list all of the current tasks jobs including the spooler task.

Do you mean you wan't to get a list of currently executing/executed tasks? If so, I'm afraid such feature is not implemented neither in uwsgi API nor in this package. You can contribute :) or you may look at celery framework.


Refactor core Asset Manager Workers

This refactors the core Asset Manger workers into services making it clearer what their responsibilities are. This should hopefully make the code easier to understand to new people.
The next part will be to change the attachment workers into services, which will make the code clearer but also reduce performance by not requiring a database look up for each subworker in the data update worker. I've decided to do that in a separate PR to make them easier to review.
Trello Card

@barrucadu Tests pass on this now.

I always feel a bit sad when I refactor something to be clearer, and the number of lines goes up.

Haha yeah, same... maybe after my next refactor it'll go down...


AzureRmWebAppDeploymentV4 no change always on (general settings)

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name:
Azure Deploy App Service task version 4
Environment

Server - Azure Pipelines

Agent - Hosted

Private Windows, agent version 2.173.0
Issue Description
When I use task version 4 "Azure Deploy App Service" in Azure DevOps, don't change the option always on for true in blade general setting to application in Portal Azure, but is created the variable alwayson with value true in blade application settings in the portal.
Configuration my task group:

After execution pipeline, see the applicaiton in portal Azure:

With task version 3, the issue don't occours:

Error logs
Log deploy task version 4(with the issue):
2020-09-28T19:33:08.2925563Z ##[section]Starting: Azure App Service Deploy: dev-wa-xxxxxxxxxxxx
2020-09-28T19:33:08.3126110Z ==============================================================================
2020-09-28T19:33:08.3126499Z Task         : Azure App Service deploy
2020-09-28T19:33:08.3126988Z Description  : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2020-09-28T19:33:08.3127452Z Version      : 4.175.2
2020-09-28T19:33:08.3127700Z Author       : Microsoft Corporation
2020-09-28T19:33:08.3128036Z Help         : https://aka.ms/azureappservicetroubleshooting
2020-09-28T19:33:08.3128395Z ==============================================================================
2020-09-28T19:33:09.8276215Z Got service connection details for Azure App Service:'dev-wa-xxxxxxxxxxxx'
2020-09-28T19:33:10.6017826Z Trying to update App Service Application settings. Data: null
2020-09-28T19:33:10.8455787Z App Service Application settings are already present.
2020-09-28T19:33:32.5432580Z Package deployment using ZIP Deploy initiated.
2020-09-28T19:34:36.3925744Z Deploy logs can be viewed at https://dev-wa-xxxxxxxxxxxxx.scm.azurewebsites.net/api/deployments/7ef604ea281f47c2860f338677fdf61f/log
2020-09-28T19:34:36.4613433Z Successfully deployed web package to App Service.
2020-09-28T19:34:36.4635348Z Trying to update App Service Application settings. Data: {"alwaysOn":"true"}
2020-09-28T19:34:37.5577059Z Updated App Service Application settings and Kudu Application settings.
2020-09-28T19:34:47.4753269Z Updated App Service Application settings and Kudu Application settings.
2020-09-28T19:34:47.8856322Z Executing given script on Kudu service.
2020-09-28T19:34:49.1030385Z Standard output from script:
2020-09-28T19:34:49.1030778Z REPLACING_FILE_SETTINGS:
2020-09-28T19:34:49.1030976Z
2020-09-28T19:34:49.1031296Z D:\home\site\wwwroot>if exist web.release.config (move /y web.release.config web.config )
2020-09-28T19:34:49.1031560Z
2020-09-28T19:34:49.1032679Z D:\home\site\wwwroot>if exist appsettings.release.json (move /y appsettings.release.json appsettings.json )
2020-09-28T19:34:49.1033435Z         1 file(s) moved.
2020-09-28T19:34:49.1033705Z
2020-09-28T19:34:51.9727564Z Successfully added release annotation to the Application Insight : dev-appInsights
Log deploy task version 3(with success):
2020-09-28T19:47:19.1106625Z ==============================================================================
2020-09-28T19:47:19.1107018Z Task         : Azure App Service deploy
2020-09-28T19:47:19.1107457Z Description  : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2020-09-28T19:47:19.1107879Z Version      : 3.175.3
2020-09-28T19:47:19.1108127Z Author       : Microsoft Corporation
2020-09-28T19:47:19.1108508Z Help         : https://aka.ms/azureappservicetroubleshooting
2020-09-28T19:47:19.1108868Z ==============================================================================
2020-09-28T19:47:29.6232626Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6233890Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6234287Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6234796Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6235176Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6235540Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6235915Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6236285Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6237444Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6238029Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6238478Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6238892Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6239321Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6239788Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6240289Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6240707Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6241132Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6241543Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6242396Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6242758Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6243130Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6243493Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6243866Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6244223Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6244597Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6244959Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6245330Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6245687Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6246060Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6246416Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6246772Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6247143Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:29.6918891Z Got connection details for Azure App Service:'dev-wa-xxxxxxx'
2020-09-28T19:47:30.6309945Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:47:31.5810079Z XML variable substitution applied successfully.
2020-09-28T19:47:35.0927010Z Updating App Service Application settings. Data: {"MSDEPLOY_RENAME_LOCKED_FILES":"1"}
2020-09-28T19:47:36.0900654Z (node:15832) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-09-28T19:48:14.2915336Z Updated App Service Application settings and Kudu Application settings.
2020-09-28T19:48:14.2915764Z Rename locked files enabled for App Service.
2020-09-28T19:48:14.4874648Z [command]"C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package='E:\Default-Pool\01_work\r281\a\temp_web_package_2925095583664996.zip' -dest:contentPath='dev-wa-xxxxxxx',ComputerName='https://dev-wa-xxxxxxx.scm.azurewebsites.net:443/msdeploy.axd?site=dev-wa-xxxxxxx',UserName='$dev-wa-xxxxxxx',Password='***',AuthType='Basic' -enableRule:AppOffline -enableRule:DoNotDeleteRule -userAgent:VSTS_280dabab-7181-47fd-87df-5effae4cbb8f_Release__3934_13049_2
2020-09-28T19:48:15.0256152Z Info: Using ID '78ac9ade-ce14-4811-b853-e329bbb7eed5' for connections to the remote server.
2020-09-28T19:48:25.2438569Z Info: Using ID '311b498e-e9d8-49a0-abb8-7dfb908b67cb' for connections to the remote server.
2020-09-28T19:48:28.9923116Z Total changes: 241 (1 added, 0 deleted, 240 updated, 0 parameters changed, 52090764 bytes copied)
2020-09-28T19:48:29.0653552Z Successfully deployed web package to App Service.
2020-09-28T19:48:29.0661845Z Updating App Service Configuration settings. Data: {"alwaysOn":"true"}
2020-09-28T19:48:30.5347345Z Updated App Service Configuration settings.
2020-09-28T19:48:31.0047749Z Executing given script on Kudu service.
2020-09-28T19:48:39.3126179Z Standard output from script:
2020-09-28T19:48:39.3126511Z REPLACING_FILE_SETTINGS:
2020-09-28T19:48:39.3126674Z
2020-09-28T19:48:39.3126982Z D:\home\site\wwwroot>if exist web.release.config (move /y web.release.config web.config )
2020-09-28T19:48:39.3128181Z
2020-09-28T19:48:39.3128526Z D:\home\site\wwwroot>if exist appsettings.release.json (move /y appsettings.release.json appsettings.json )
2020-09-28T19:48:39.3128869Z         1 file(s) moved.
2020-09-28T19:48:39.3129010Z
2020-09-28T19:48:41.1188985Z Successfully added release annotation to the Application Insight : dev-appInsights

I have the same issue, so I'm using v3 that works fine...

@andressantos10 You have added "always on" property to in appsettings for task 4 add it in configuration settings same as what you have done in task 3.

@AmrutaKawade  you didn't understand the issue! If you use task 4 version, this option changes the application settings blade in configuration option in the app in the portal, creating a variable "alwayson" true.
If you use task 3 version, it correctly changes the "alwayson" field to true blade general settings  in the app inthe portal.
See the images attached in descritption this case.

@andressantos10 I meant Check below configuration of your task. "always On" is added in App Settings box instead add it in Configuration Settings box

@andressantos10 Let me know if above solution works

Closing this issue as its task configuration is not correct. Please try out above solution. Feel free to reopen if solution doesn't work


Add support for protecting URL's being authentication fixes #34

Ok I pushed all these changes. @pushyamig and you'll have to set
STUDENT_DASHBOARD_SAML=false
Now in your env or you'll be blocked from using this locally.


Allow updating to internal link

Allow updating a link to a private link. See https://github.com/owncloud/ocis/issues/4979

@micbar could you check my solution? It seems to a problem of private links vs. denials. Not sure my approach is the perfect one

@micbar needed some more changes to make this work. Please double check

I cannot get the PR to work locally. The alias link cannot resolve. I am missing the alias=true prop in the response.

@micbar I needed to pass an islink parameter to the RoleFromResourcePermissions func. That's not nice but it is the only way we can differentiate between "denial" and "internal link".
Please recheck and retest

Please recheck and retest

Works fine now.


Removing the perl dependency after #2.

Ping @ocefpaf. Was perl necessary, or was it to workaround another issue? I've tried removing it from here to find out.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

So PERL most emphatically is a build dependency. I now am checking to see whether it is a run dependency.
Incidentally, just add perl as the dep, and run conda-smithy rerender 😉

makeinfo -h also needs perl. There we go. 👍


Modify adversarial attacks loss

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context / Related issue

How Has This Been Tested (if it applies)

Checklist

[ ] The documentation is up-to-date with the changes I made.
[ ] I have read the CONTRIBUTING document and completed the CLA (see CLA).
[ ] All tests passed, and additional code has been covered with new tests.

@LaurentMnr95 please merge master and if tests pass you can merge

@LaurentMnr95 please merge master and if tests pass you can merge

@LaurentMnr95 please merge master and if tests pass you can merge

Sadly, there is no simple hack for merging master directly in the github interface, so you have to manually git pull origin master or git rebase master or something like that.

@LaurentMnr95 please merge master and if tests pass you can merge

Sadly, there is no simple hack for merging master directly in the github interface, so you have to manually git pull origin master or git rebase master or something like that.


feat: display topic details using /api/v1/topics/{id} call

Description:
Leverage the new mirror-node /api/v1/topics/{id}  REST API to display topic details.
The 3 properties mandatory in the API result are always displayed: Memo, Valid from and Valid until.
The other (nullable) properties are shown only when their value is non null.
Related issue(s):
Fixes #713
Notes for reviewer:
API available only on previewnet for the time being.
Deployed on staging server.
Checklist

[ ] Documented (Code comments, README, etc.)
[x] Tested (unit, integration, etc.)

Closing this PR since it is stuck by the DCO check. Will re-open a clone of this one.


Drop internal version of extensions/pkg/apis/config API

How to categorize this PR?

/area dev-productivity
/kind cleanup
What this PR does / why we need it:
This PR drops the internal version of the extensions/pkg/apis/config API.
There wasn't even any conversion code and we only used the internal version. So get rid of the duplication to prevent confusion and reduce development/maintenance efforts.
Which issue(s) this PR fixes:
Part of https://github.com/gardener/gardener/issues/11043
Special notes for your reviewer:
Release note:

The package `github.com/gardener/gardener/extensions/pkg/apis/config` has been dropped. Use the versioned variant of the package instead: `github.com/gardener/gardener/extensions/pkg/apis/config/v1alpha1`.

/test all

/retest

/lgtm

From the PR description: It was probably meant to say that we're only using the external version, right?

This PR drops the internal version of the extensions/pkg/apis/config API.
There wasn't even any conversion code and we only used the ~internal~ external version.

We always used the internal version before this change. From now on, there is only the external version and we switch all usages to the external version.


Check the first and last name arguments in add_pii().

The add_pii() function takes arguments for first_name and last_name, both default to None.
These arguments must both be None or not None simultaneously.
If first_name is None and last_name is not None (or vice-versa) throw an SDKError.

Coverage decreased (-0.5%) to 82.913% when pulling 98c08e9263572b218e44172b4fc596e1fd6088c4 on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage decreased (-0.5%) to 82.913% when pulling 98c08e9263572b218e44172b4fc596e1fd6088c4 on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage decreased (-0.5%) to 82.913% when pulling 98c08e9263572b218e44172b4fc596e1fd6088c4 on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage decreased (-0.5%) to 82.913% when pulling 504a1f54ba008b715d69c4840f4ed1f1d1847254 on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

@mattgeiger Ok, np

@mattgeiger are you satisfied with the PR now?

Coverage decreased (-0.3%) to 83.142% when pulling 66f9f3f7bf53c308cb9750678b9424ff5b28d765 on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage decreased (-0.3%) to 83.142% when pulling 5d40339a0227fc95e85222db82fd627c1afe22e5 on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage decreased (-0.3%) to 83.142% when pulling 14c4c1ad5408c84786a7d828bb728138f2d53d6a on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage decreased (-0.08%) to 83.372% when pulling c75c6d8373e5bd9a2c7d99f42f116ed0fbe0e61f on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.

Coverage increased (+0.3%) to 83.716% when pulling ad8c9be5b4f94441a1a81bf2b700b0dd3214fa3e on bugfix/check-first-and-last-name-when-adding-pii into ac9cbbc3142def263944f57e57587a5be5fffacd on master.


Allow the abilty to add custom, private wordlists during builds

It would be really nice to have the ability to add custom, private wordlists when building a new container image.
In commit 57ba272382, wordlists were fetched into the working directory as a git submodule before the docker build step.  This left the option open to add new custom, private wordlists just before the container image gets built.  This feature was removed by commit 1187f03b0c7.
Once https://github.com/keepassxreboot/keepassxc/issues/2854 is resolved, new wordlists could then be added after the container image build stage.

https://github.com/keepassxreboot/keepassxc/issues/989


Merges 2.2.0

to: @ryandeivert
cc: @airbnb/streamalert-maintainers
Background
Merges our release branch, release-2.2.0, into master.
Changes
Alert Publishers
#900
Alert Publishers is a new feature designed to allow for more customizable rendering of Alerts in their specified outputs. This allows StreamAlert rules to execute python code to transform, hydrate, or filter out Alert fields prior to being dispatched to alert processors. It can even be used to customize the look & feel of alert payloads.

Coverage decreased (-0.4%) to 96.642% when pulling 26a7af27b13633fce864c640baad4a56c59268f4 on release-2-2-0 into 0f0bf3da2fa6407ff3dd58f93fd15d21c0a3c50f on master.


Dataflow GCS streaming file sink uses runner determined sharding

Integrate file sink with shardable GroupIntoBatches (BEAM-10475) to allow runner determined dynamic sharding for streaming use cases.
Imported from Jira BEAM-11934. Original Jira may contain additional context.
Reported by: sychen.

Runner based sharding for GCS exists (done during the runner translation layer), though additional support for the runner backend to support it would require updates to the Beam model (proto API) to differentiate between the two cases.


Any way to also get "off market" homes?

If you search in Zillow for a house address that is not listed as sale/rent/sold you will get a page with info. Is there a filter that can be used to also pull those in by lat/lon?

no, in order to know if it was sold you first need to save all the results in a database, then iterate over all this results each day and check if the property still available

So for properties that have never been listed on Zillow they are not accessible aside from searching the address manually?

if you would have to do it manually on the page, how would that look like?

You have to search for an address (or zoom way in and click the dollar value bubble on the house) which brings up its listing. However the status is Off Market and the tooltip says "This home is not currently listed for sale or rent on Zillow."
It doesnt appear in searches as searches seem to require a status of For Sale, For Rent, or Sold

you are right, but it looks like it does return the properties off the market , you can try with any rent, sold or for sale, you will still get the off market properties


How to: Restore to a Point in Time - Recommended Version link entirely un-helpful

Starting at https://docs.microsoft.com/en-us/previous-versions/sql/sql-server-2008-r2/ms190982(v=sql.105)?redirectedfrom=MSDN and clicking on Recommended Version link brings user to list of generic high level links: https://docs.microsoft.com/en-us/sql/?view=sql-server-ver15.
Same is true for  Recommended Version links in pages having same banner at top starting with: "This content has been retired and may not be updated in the future."
This on page showing: How to: Back Up a Transaction Log (SQL Server Management Studio):  a common practice

The right link maybe https://docs.microsoft.com/sql/relational-databases/backup-restore/restore-a-sql-server-database-to-a-point-in-time-full-recovery-model?view=sql-server-ver15
@markingmyname @rothja Maybe you could see if this is fixable with a redirect or something.

The right link maybe https://docs.microsoft.com/sql/relational-databases/backup-restore/restore-a-sql-server-database-to-a-point-in-time-full-recovery-model?view=sql-server-ver15
@markingmyname @rothja Maybe you could see if this is fixable with a redirect or something.

#assign:rothja

#assign:rothja

@engfob My apologies for the delay on this. I will be following up soon. Thanks for your patience and your feedback!

@engfob I apologize for the very long delay in my follow-up. I did some research on this and found that it seems that "Recommended Version" link is just hard-coded to the current SQL Server documentation set but not specific to this article. That does seem to be confusing, and I will try to share this feedback with our engineering team. In this case, you have to search for the similar article in our current docs. For example, I searched for your first article and found this equivalent in the current SQL Server documentations set Restore a SQL Server Database to a Point in Time (Full Recovery Model). I'm sorry for the confusion!
#please-close

Thanks for the reply Jason.  I would call this inaccurate vs. confusing.  I have found this on many pages already.  I would propose a change from ‘Recommended Version’ to ‘General Information’, if these links are not going to be re-directed accurately.
From: Jason Roth @.>
Sent: Friday, May 14, 2021 3:42 PM
To: MicrosoftDocs/feedback @.>
Cc: engfob @.>; Mention @.>
Subject: Re: [MicrosoftDocs/feedback] How to: Restore to a Point in Time - Recommended Version link entirely un-helpful (#3409)
@engfob https://github.com/engfob  I apologize for the very long delay in my follow-up. I did some research on this and found that it seems that "Recommended Version" link is just hard-coded to the current SQL Server documentation set but not specific to this article. That does seem to be confusing, and I will try to share this feedback with our engineering team. In this case, you have to search for the similar article in our current docs. For example, I searched for your first article and found this equivalent in the current SQL Server documentations set Restore a SQL Server Database to a Point in Time (Full Recovery Model) https://docs.microsoft.com/en-us/sql/relational-databases/backup-restore/restore-a-sql-server-database-to-a-point-in-time-full-recovery-model?view=sql-server-ver15 . I'm sorry for the confusion!
#please-close
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MicrosoftDocs/feedback/issues/3409#issuecomment-841462614 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AQPWQLKDWD7I5NM62TJKKYDTNV4HNANCNFSM4WO7YYNA .  https://github.com/notifications/beacon/AQPWQLPTJYTGONZGXGTC7PDTNV4HNA5CNFSM4WO7YYNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGIT3GVQ.gif

@engfob I think this is an excellent suggestion. I will pass this along to our engineering team. Thanks, and sorry again for the confusion!


adding a new many-to-one field copies validation classes as well

Right now if you have an invalid input on a many-to-one form and click to add a new one, the validation class copies as well. Since we are clearing all input values, we should also clear up any validation classes.

@mapsam I wrote a somewhat generalized jQuery plugin to do the copying. It might be helpful to you.

Thanks for the snippet, @bsmithgall!
This is no longer an issue since we are now using a hidden "clone" for adding new fields, and not copying pre-existing inputs.


Fix n_jobs problem in LogisticRegression

why is n_jobs not supported / had to be fixed ? It would be good to raise a warning for users to be aware of it also ...

@PetrovKP @OnlyDeniko
Note that the docs say that all parameters can be used: https://intel.github.io/scikit-learn-intelex/algorithms.html they should clarify that n_jobs cannot