Reference Error : xxx is not defined for object declared in external resources

Since commit https://github.com/qooxdoo/qooxdoo-compiler/pull/654 (I tried to install qooxdoo-compiler with version 1.0.0-beta.20200415-0654 and 1.0.0-beta.20200420-2025 and the bug only appears since the second one) all the object i declared in external resources are not recognized by my application and so it doesn't work at all.
This problem only appears in Build Target not in Source Target.
My external resources are javascript libraries placed in folder source/resource/script/, and declared in Manifest.json in "externalResources"/"script", with this path : "script/mylib.js"
Don't mind to ask more details if it can help you.

Does it work if you declare the asset  with @asset(script/*) in the header comment of the class where the script is used?

I tried with @asset(script/*) in the main class of my application, however i didn't tried in the class were i'm using the script

I just tested to add @asset(script/*) in all the files where my libs are used but it doesn't change anything always the same error.

I use the same construct in my manifest.json. Everything is working. Could you have a look into your index.js if you found an urisBefore array with your external files in it?

The urisBefore seems to be defined in index.js however i still have an undefined reference when i use the object declare in the files:
 urisBefore:["../resource/script/mylib1.js","../resource/script/mylib2.js"]

Are they loaded in index.js?

How can i see that? should the entire files be in index.js?

No the external files are loaded here:
init: function(){
var l = qx.$$loader;
l.decodeUris(l.cssBefore, "resourceUri").forEach(function(uri) {
loadCss(uri);
});
allScripts = l.decodeUris(l.urisBefore, "resourceUri");
if (!l.bootIsInline) {
l.parts[l.boot].forEach(function(pkg) {
var add = l.decodeUris(l.packages[pkg].uris);
Array.prototype.push.apply(allScripts, add);
});
}

Hum it seems to be ok, i have the same code of you, when i try in dev console to make the decodeUris and then i looked at the result it is a correct array of all my script path (i try to put the path in my web browser and i see correctly the code of my lib), however i don't know if there is a better way to check if the code is loaded.

However the scripts are not included in the head of my page, and when i run manually the commands (decodeUris then loadScript) the files appears in head of the page (don't know how it is supposed to be in normal way)

loading is done here
function loadScript(uri, callback) {
var elem = document.createElement("script");
elem.charset = "utf-8";
elem.src = uri;
elem.onreadystatechange = elem.onload = function() {
if (!this.readyState || readyStateValue[this.readyState]) {
elem.onreadystatechange = elem.onload = null;
if (typeof callback === "function") {
callback();
}
}
};

And they are shown:

On my project, i have only the (index) and index.js, all the script doesn't appear (since the update). (They appeared when i run qx.$$loader.init() manually from the console (with an error however it think it's only because it's not a normal way to use it).

However i'm trying to reproduce the bug in a sample app and can't manage to do it, i have to try what part of my apps provoke the bug.

A sample app with the bug if needed : https://github.com/rommni/testScriptIncludeQooxdoo
Just clone it and do npx qx serve --target build to see the bug.
(If you do npx qx serve the app works)

Found the problem:
With build target all of your own code is merged in one big javascript file index.js.
You try to load your TestScript direct in the init section of your property nameTest. This is done during file loading.
Loading of your own script is done later.
So you have 2 possibilities:
- put loading of TestScript into your class constructor.
- change your index.html in boot dir so that your own script is loaded before index.js. Using externalResources/script is not working in this case.
The whole thing works in source because there each class file is loaded seperately.

ok so if i understand well the externalResources shouldn't be used in the init of qooxdoo properties?
Will try to change my code accordingly, thank you for your investigation.

Yep

@rommni : Can we close this issue?

Yep thank you I found an other way to write my code which solve the issue


Odd tiling behaviour

We tried using OpenLayers 3 for the first time for a video game map, but it seems to only render a part of the image everytime.
See this detailed post: http://stackoverflow.com/questions/35928163/openlayers-3-static-tiles-and-xyz-coordinates
Here's the JS Fiddle for faster access.
What do you think?

Here's the [JS Fiddle](JS Fiddle) for faster access.

The question is better asked on Stackoverflow, like you did already.
There is nothing wrong with OpenLayers. You are using inappropriate tools. Use ol.source.TileImage instead of ol.source.XYZ, and set its tileGrid appropriately by using the ol.tilegrid.TileGrid constructor with resolutions and origin.


[doc] add shardformer user document and update booster plugin document

Update Booster Documents: #4669

Add Shardformer User Document: #4702

Add Support Matrix for Shardformer Document: #4728


When printing logging, dump out the whole function.

Just to make it a little quicker to debug/get this information when debugging the pass. I have been wanting this and just hadn't gotten around to adding it. It just centralizes the last piece of information that one wants to reach for when debugging.

@swift-ci smoke test


Failed building wheel for zlib-ng

This happened installing on M1 Mac on macOS 14.1.2 using a virtual environment.
Is there a general issue?

What was the actual error it did give?

I only got this, but I will check again and report back.

Fixed for me by installing make:
brew install make


Lower fan speed options

Hello, I have an X1 Yoga 7th Gen with the i7-1260P processor and it seems that even on its lowest setting the fan spins at around 5k RPM and is easily audible in a quiet room.  Would it be possible to set lower RPM values?  I believe a slower fan speed could keep the CPU cool under low loads without being audible.  As it is I wait until 70C to turn the fans on and then they quickly cool the CPU back to the 40C range.  Being able to spin the fans slowly and silently should stabilize the CPU temp and keep the system quiet.
Ideally maybe go for 1k RPM and then move up from there.  Perhaps with more steps than 7 or have the 7 "standard" steps fall in above the slower ones.
Being able to lock the fans on speed 1 is still an improvement over Lenovo's logic.  It appears to try to keep the CPU in the 40C range at all times spinning up the fans to high speed quickly.

Hi
The steps are actually defined by lenovo in their EC, which is also defined in their firmware, fan control is done by setting the address in the EC to a value that corresponds to the fan speed, so it cannot set the fan speed directly.
Maybe you could check if your BIOS firmware is up to date, or try fan level 17 (0x11 in hexadecimal), it should be equal to the lowest possible fan speed, but perhaps it will have a lower speed on your laptop.
On most ThinkPads there should also be an option in the BIOS to adjust adaptive thermal management scheme under power, maybe tweaking that will help

I was afraid of that.  I tried 17/0x11 but it seems ~5k is the lowest speed demanded by the controller.  To add to the loud fans it seems Lenovo's CPU boost logic is quite aggressive attempting to boost to > 4GHz frequently pushing temps up and requiring the fan to spin up.  I've played with the BIOS settings and it only allows "Balanced" or "Performance".   There is also an "Intelligent Cooling Boost" options but it does not appear to help either.  I supposed it'll have to be a hardware mod at this point.  I wonder if the fans are still PWM controlled.  Maybe I can lower the PWM voltage to spin the fans down.  Thank you anyway for a fantastic tool!

Just what I've been trying to do as well - the fan on the previous gen 6 with top of the range i7 has a lower minimum RPM and only comes on at 70C.  it also ideals around 40C which is a massive difference to the i7-1260P which idles at 55-60C
This combination of a CPU which consumes ~2x more power at the same load as the 11th Gen and emits more heat combined with poor thermal control is a step backwards for Intel and Lenovo.  I have had to disable Turbo and increase the Speed Shift using ThrottleStop just to keep it from being too annoying day-to-day...
If you find anything else out, please let me know!

Just what I've been trying to do as well - the fan on the previous gen 6 with top of the range i7 has a lower minimum RPM and only comes on at 70C. it also idles around 40C which is a massive difference to the i7-1260P which idles at 55-60C
This combination of a CPU which consumes ~2x more power at the same load as the 11th Gen and emits more heat combined with poor thermal control is a step backwards for Intel and Lenovo. I have had to disable Turbo and increase the Speed Shift using ThrottleStop just to keep it from being too annoying day-to-day...
If you find anything else out, please let me know!
EDIT - looks like a re-paste of the thermal heat glue helps a significant amount. See here - https://www.reddit.com/r/thinkpad/comments/zlfqm9/thinkpad_x1_x1_yoga_with_the_i7_1260p1280p_heat/

Replacing the existing compound with arctic silver 4 was my first attempt to quiet things down.  There was good heat transfer from the CPU to the as I could keep the fan on speed 1 with good cooling but of course you're still running the fan.
My ultimate solution is aggressive.  I removed a section of the black material over the CPU, placed a total of four 1x10x10mm copper shims with AS4 on both sides on top of the heat pipe and replaced the back plate.  This allows heat to directly transfer from the heat pipe to the back cover.  Then I've situated an AIO waterblock under the laptop on that area of the case also with AS4.  At this point I can run the CPU at max TDP without hitting the thermal limit and without running the fan.  ThrottleStop can set the TPL.  By default it's 15 and 20 for pl1 and pl2.  I now have it set at 28 and 65.  Even doing benchmarking or stress testing will not thermal throttle the CPU.  It primarily hovers between 37 and 40 C now.  I'm probably going to 3D print a laptop stand that will hold the cooler and radiator in the right spot.  When I do that I'll also see how the cooling is with just air flow and stock PL1/PL2 settings.  Will probably also try a thermal pad instead of AS between the water block and the laptop case so I don't have to clean it when traveling.   I did the AS because I had spent weeks of trying different things and just wanted a best case scenario.
Things I found that impacted the throttling profile
BIOS Setting for cooling profile
BIOS Setting for CPU power saving
Windows advanced power settings
Processor power management -> Processor performance autonomous mode
This is hidden by default and must use a registry edit to show it but it had the largest impact on the throttling profile.  Where the defaults have the processor boost very aggressively and use 1.2 - 1.3 volts setting this to disabled in conjunction with the BIOS cooling profile seems to use a much more reasonable boost profile.  Unfortunately none of the other clock related settings under PPM tree had any effect on the profile.
ThrottleStop
I still couldn't get control of the clock multiplier or voltage but I was able to modify the Turbo Power Limits (TPL).  Setting these lower will reduce the target TDP of the processor for long (PL1) and short (PL2) power usage and keep it a little cooler though I could never get it low enough for the fans to stay off.  The heat pipe system simply can't dissipate heat without the fan running.  Using settings of 15 & 18 respectively and setting the fan to level 1 kept the temp in the 40s until you put it under a heavy load but obviously with a performance decrease.
On the other hand if you increase these it will easily overheat the CPU so do this with caution.

Thanks very much for the reply & info, glad to see you found a solution!
Water cooling is a little extreme and expensive for me :), will try a re-paste and see, then perhaps the copper plates.
On my back plate, there is a 'pad' underneath the black material right above the heat sink - did you find what this was (perhaps a thermal pad?)


RPI4 led name changed from led0 to ACT?

I don't know what and when this changed, but led0 no longer exists.
Instead we have PWR and ACT.

The only reason this could change is via DeviceTree update. This brings me to my 2nd point -- since we don't fully control DT for RPi4 I suggest you actually add a block instead of replacing it so we don't ruin those units that may still be working fine with older DTs.

But wouldn't a change in the DT come with a new version of EVE?
It isn't like I changed the DT intentionally.
And since this is a raspberrypi.rpi.raspberrypi,4-model-b.brcm,bcm2711 (and not a different model), if we want to support multiple options we'd have to list multiple LEDs and have ledmanager try the listed LEDs until it finds one which exists. Is that a reasonable approach?

I can see some recent changes of dts inside raspberry/linux fork. https://github.com/raspberrypi/linux/commit/22201d4868f66aa7773d5c4bad7bab1795ee6757#diff-2401e76a6c909c36337ef376d13ec770c079dc46ee9394cc52dce7eacd2d36cf

I can see some recent changes of dts inside raspberry/linux fork. raspberrypi/linux@22201d4#diff-2401e76a6c909c36337ef376d13ec770c079dc46ee9394cc52dce7eacd2d36cf

But that doesn't have a label of "ACT" for "led0". I have to look at what we have now in EVE as DTS.

Hey @eriknordmark -- is this still relevant or can we close this?

The reason for the ACT label it's because we get the device tree file for RPi from u-boot, and following the includes in the device tree, we reach to the following line:
https://github.com/u-boot/u-boot/blob/v2022.10/arch/arm/dts/bcm2835-rpi.dtsi#L8
That defines the label ACT. So the LED must be renamed on EVE. However, I agree that must be backward compatible (for the reasons already pointed out here).

@rene  it's been a while and i dont have my rpi with me. Can you check that this pr still fixes it?

@rene it's been a while and i dont have my rpi with me. Can you check that this pr still fixes it?

@eriknordmark , sure, we have 3 RPis in the office, I will check next week... no worries...

@eriknordmark , I have reworked this PR and addressed all my own comments :smile:
I tested with a Raspberry PI 4 and the LED it's working as expected. I think you can fetch my branch https://github.com/rene/eve/tree/rpi-fix-led and push to yours (and of course, review my changes)...

@eriknordmark , I have reworked this PR and addressed all my own comments smile I tested with a Raspberry PI 4 and the LED it's working as expected. I think you can fetch my branch https://github.com/rene/eve/tree/rpi-fix-led and push to yours (and of course, review my changes)...

@rene easier if you submit a PR from your branch and I can review and approve it.

@eriknordmark , here is the new PR: https://github.com/lf-edge/eve/pull/3067


Add missing protobuf serialisation functionality GetIndexedFieldExpr.

This is required in order for ballista to serialise the indexing expressions for nested lists. Otherwise, ballista cannot execute any SQL that contains indexing expressions.
Which issue does this PR close?

Closes https://github.com/apache/arrow-datafusion/issues/5323
Rationale for this change
Add missing protobuf serialisation for GetIndexedFieldExpr that is required when running queries that index nested structures.

What changes are included in this PR?
Add protobuf serialisation for GetIndexedFieldExpr and a unit test.

Are these changes tested?
Yes

Are there any user-facing changes?
No

I merged this soon as I don't think it is controversial. I am happy to make changes or back it out if any maintainer feels differently


[SIS-032] Wifi warning and unsaved changes overlays

Description
Created overlays for wifi warnings and unsaved changes.

Ticket #32
Type of change

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My PR title is formatted as [SIS-<issue number>] <issue name> (eg. [SIS-010] Create UI button)
[x] I have linked my issue/ticket on the project board to this PR
[x] I have sent my PR to #team-sistema-prs on Slack and requested two people to review
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation (if required)
[x] My changes generate no new warnings
[ ] I have tested that my feature works as expected on Android and iOS. If you are a Windows dev, please request a Mac dev to run your branch. If it's a front-end change, I have attached screenshots of what the result looks like on both platforms.

i love how github says there are merge conflicts, i go to fix on my laptop and BOOOM no conflicts


Allow for non-SSL installation for reverse proxy users

This project looks to be a sorely needed integration for Home Assistant and I have been watching the vast amount of improvements that have been made in a short amount of time.
I can install this add-on without issue, but because I run Nginx outside of Home Assistant, no certificates are stored on my Home Assistant machine.  If I manually add the certificates to the /ssl directory, the add-on will start and I can add the integration, but any attempt at connection results in SSL errors.
If there is a way to get this add-on to work without local SSL certs, that would be greatly appreciated as I use reverse proxy for just a couple of machines.

Agreed. I suggest we add a new option like:

[ ] SSL

Which can be deactivated in the configuration.

@thewolfman56 version 1.2.0 contains an option to disable any kind of TLS/HTTPS/SSL. Would you mind testing it in your reverse proxy setup? I would be extremely grateful.
Here is a guide on how to configure NGINX to proxy Asterisk's WebSockets interface:
https://warlord0blog.wordpress.com/2020/04/16/asterisk-webrtc/

Ok, I had to revert the feature added in 1.2.0, and now 1.3.0 contains an option called generate_ssl_cert, which comes disabled by default, but you probably want to enable it.
Then, you can configure your reverse proxy to proxy https://<ha-ip>:8089.


Add a note about hiding dynamic profiles

We've had two complaints about this in the span of a week. Maybe we need more documentation?

[x] Closes microsoft/terminal#8270
[x] I work here

Docs Build status updates of commit 40effb1:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

TerminalDocs/troubleshooting.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

@zadjii-msft there are a couple typos in your commit

Docs Build status updates of commit 71b8d42:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

TerminalDocs/troubleshooting.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 71b8d42:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

TerminalDocs/troubleshooting.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Fixing my name

Deploy preview for youthful-colden-6bf88b ready!
Built with commit 63cc4af5ec859064a1a70aa72a7420bd8985bab1
https://deploy-preview-62--youthful-colden-6bf88b.netlify.com


KAFKA-14293: Basic Auth filter should set the SecurityContext after a…

… successful login

Added the AUTHENTICATION Priority to the JaasBasicAuthFilter. This way if multiple filters are present, they will be processed in the right order.

Extended the filter to set the Security Context of the request in case of a successful login. The filters sets the user, the authentication type and and the isSecure property indicating whether this request was made using a secure channel, such as HTTPS.

Extended the JaasBasicAuthFilterTest test with a new test to see if the security context is set correctly

Previous test are working with the new change

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@rhauch @ewencp Could you have a look? Thanks!

@C0urante can you take a look when you have some time? Thanks!


Add delegates outside of the Worker using a :via flag

Based around #8.
Right now the Worker has the code for various functions which delegate in order to notify hooks correctly. With this PR we can delegate in the main interface and maintain the hook notifications using :via.
The example from the issue is if you call something like Cachex.incr(:my_cache, "key", via: :decr), it notifies using { :decr, "key" }. After this PR is merged, this will function correctly. The same implementation exists for both expire_at/4 and persist/3.

Coverage remained the same at 100.0% when pulling 8411411f80dbe99e0559e5e80ee0089619787725 on issue-8 into caa02709461bf034bf94d49f80582dc09a053f97 on master.


Patch version update CI.

Description
Adds a new CI job to update the patch version of the helm charts. This job is trigger by an event send from other Kubewarden repositories (e.g. kubewarden-controller). This CI job will sync the CRDs when necessary as well as update the container image tag used. The container image tag is the version sent from the event payload sent by the other repositories.
Related to https://github.com/kubewarden/kubewarden-controller/issues/352

@flavio, I've updated the script to update the annotations versions and fixing the issues that you reported in your last review. I just need to talk with Olivier to find a solution to an issue after we decided to use PR to merge the changes.

@flavio @olblak these are PRs created by the script from this PR:
Patch update with CRD update: https://github.com/jvanz/helm-charts/pull/31
Patch update with no CRD update: https://github.com/jvanz/helm-charts/pull/32


Question

Hi, is this code available on maven repository?

Not yet, but I'll work on that.

Gradle
compile group: 'com.victorlaerte', name: 'jfx-asynctask', version: '1.0.1', ext: 'pom'

Maven
<!-- https://mvnrepository.com/artifact/com.victorlaerte/jfx-asynctask -->
<dependency>
    <groupId>com.victorlaerte</groupId>
    <artifactId>jfx-asynctask</artifactId>
    <version>1.0.1</version>
    <type>pom</type>
</dependency>

Thanks @khyrulimam


Update Post “best-way-to-treat-your-home-for-termites-white-ants”

Automatically generated by Netlify CMS

👷 Deploy Preview for chic-shortbread-66a66b processing.

Name
Link

🔨 Latest commit
884d95d7a2a536d1ecb878aa7e918d5ee0c0ad31

🔍 Latest deploy log
https://app.netlify.com/sites/chic-shortbread-66a66b/deploys/654ac7c3b087240008c02666


Error after "Next Scenario" in Katakoda

After completing a scenario in Katakoda, click the right arrow after "Step X of X" and you see a "Congratulations!" screen with a "Next Scenario" button. Clicking that button results in:

Coming Soon
Sorry, it looks like you've come across something not quite ready yet, bare with us.

I'll probably remove that last page if we can't customize it.  Can we @benhall ? The "next" scenario button doesn't make sense here, as it breaks the current workflow: learn-> try interactive -> quiz

Couple of options.

Set the data-katacoda-ctatext and data-katacoda-cta (for the link) attributes. You can then direct them to the quiz...
Set the attribute data-katacoda-hidefinish=true - This will disable the final finish dialog.


image: resolve abolute path from src

Related to iterative/dvc#7664

Codecov Report

Merging #39 (d9d8ed1) into main (369396f) will increase coverage by 0.10%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #39      +/-   ##
==========================================
+ Coverage   95.68%   95.79%   +0.10%
==========================================
  Files          14       14
  Lines         464      476      +12
  Branches       76       69       -7
==========================================
+ Hits          444      456      +12
  Misses         15       15
  Partials        5        5

Impacted Files
Coverage Δ

src/dvc_render/base.py
100.00% <100.00%> (ø)

src/dvc_render/html.py
70.00% <100.00%> (ø)

src/dvc_render/image.py
100.00% <100.00%> (ø)

src/dvc_render/plotly.py
100.00% <100.00%> (ø)

src/dvc_render/vega.py
100.00% <100.00%> (ø)

tests/test_image.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 369396f...d9d8ed1. Read the comment docs.

@daavoo Yes, it will work.
Hovewer, if we want to make HTML reports portable (eg copy dir and send via email) we need working relative paths. In iterative/dvc#7549 I made plots use relative paths, but relative to cwd and not index.html.


typo

Replace "differenciated" with "differentiated".

recheck

You can retrigger this bot by commenting recheck in this Pull Request

Hmm. I tried the above but maybe I'm not authorized? Just before I commented "recheck" I went through the CLA approval process. Or tried to, at least. 😅

Nevermind. Now I see a "MLCommons GitHub Access for Philip Durbin" email and a PDF to fill out. Wish me luck.


Fix linter errors.

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "lint" git@github.com:anshulpundir/swarmkit.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

LGTM


replaced most database queries in report views

including associations during reports#show
helps with issue #42

I am having issues with getting the test bed app to run - hopefully I can fix that and get the CI to pass, otherwise, I hope someone else can fix what I missed. I suspect it has to do with ReportFormatter - that class uses .where on the view level. Since @responses is no longer a database query object, it needs to be modified accordingly.


add OUTPUT clause support.

Hey 👋
output clause is supported in dialects such as MS SQL Server (MSSQL).
It works just like returning, but:

forces columns to be prefixed with inserted for new values, deleted for old values, and table names/aliases that are not the main table. Since we don't support mssql's way of adding more tables to context in all the queries output is supported in, I left it out for a future PR.
for merge queries, $action returns delete, insert or update. left it out for a future PR.
can be used to insert into a variable or another table with the into sub-clause. left it out for a future PR.

👏👏👏👏👏👏

When will this be getting released?

@igalklebanov Is this on its way to be published? I cannot find docs on how to use it.


fix https://github.com/hootlex/vuejs-paginator/issues/26

Thanks for fixing the issue with the events, @sebestenyb.
Can you explain how is this change beneficial?

Sure.
The original code overwrote the request headers even if none was set on the component, in that case with an empty object.
This causes issues with for example authenticated APIs, where the Auth token is sent in the headers. As in Laravel and Vue.
Also can be an issue if the response type is determined by the content type request header.
So this changes is the the paginator should only overwrite the headers if they're explicitly defined on the component.

Thanks for the explanation. Since the paginator handles the ajax requests by itself, it won't pass any headers unless they are specified in the headers property.
Even if it was passing the headers, I don't see how theυ are not getting overwritten now, since config var is initialized to an empty object.

The config is an empty object, but if there's no headers key on it, axios
or vue-resource will use the default headers.
This change allows to keep the default headers if none is specified on the
component.
It's the common practice in Laravel that you define
axios.defaults.headers.common or Vue.headers.common in your app.js and it
will be used in every request - except it we overwrite it on instance
basis, which happens in the paginator without this snippet.
Cheers, Sebi
On Sat, 12 Aug 2017, 09:53 Alex Kyriakidis<EMAIL_ADDRESS>wrote:

Thanks for the explanation. Since the paginator handles the ajax requests
by itself, it won't pass any headers unless they are specified in the
headers property.
Even if it was passing the headers, I don't see how theυ are not getting
overwritten now, since config var is initialized to an empty object
https://github.com/hootlex/vuejs-paginator/pull/27/commits/5b06f26c16016d1f73c944303a74b05a73ba3916#diff-35142a17e57d6f8bcaca0500775bd707R52
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hootlex/vuejs-paginator/pull/27#issuecomment-321927467,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAtXLJ4cCguESg2_Pow3Xrq0TL94nwzAks5sXM1pgaJpZM4O0JLA
.

Using axios.defaults.headers or Vue.http.headers is a good point.
Though, axios way won't work unless Vue.http === axios. This would be a nice improvement - fall back to axios if Vue.http is not defined.
Merged 🙂


Avoid unnecessary copy of the closure for runner (repeat 8834e4f)

Hello, this is very small changes that add const to the closure for Clang-tidy.
These changes just repeat the previous changes 8834e4f that was made by @mrry.

@mrry Thank you for the review.
I made this PR by using DevReplay that found recent similar changes from this repo.
If you have an interest in more potential tensorflow rules, please give me feedback.
[{
  "repository": "tensorflow/tensorflow",
  "sha": "8834e4f86c8d5adc89b990249d2eb6532f2fc03f",
  "author": "Derek Murray",
  "created_at": "2020-01-05 02:56:38",
  "condition": [
   "[](std::function<void()> ${1:Identifier}) { ${1:Identifier}(); });"
  ],
  "consequent": [
   "[](const std::function<void()>& ${1:Identifier}) { ${1:Identifier}(); });"
  ],
  "abstracted": {
   "1": "f"
  }
 }]


GltfLoader updates

This PR is extracting out some code from the model-loading branch, mostly related to glTF loading.

Rounded out some of the FeatureTexture API
Added full support for KHR_texture_transform
Improved state machine logic in GltfLoader
Added enums VertexAttributeSemantic and InstanceAttributeSemantic
Load instance attributes as typed arrays if "ROTATION" exists since quaternions need to be converted to rotation matrices on the CPU

Thanks for the pull request @lilleyse!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

Reviewers, don't forget to make sure that:

[ ] Cesium Viewer works.
[ ] Works in 2D/CV.
[ ] Works (or fails gracefully) in IE11.

@sanjeetsuhag updated!

@sanjeetsuhag updated!

Changes look good to me. Thanks @lilleyse and @ptrgags!


Add node key creation steps

https://github.com/threefoldtech/grid_deployment/tree/development/docker-compose
Only missing point is how to create key.
Generate a TFChain node key with subkey

Note: If you deploy the 3 network instances, you can use the same node key for all 3 networks. But it is recommended to use 3 different keys to facilitate management.
echo .nodekey_mainnet >> .gitignore
../../apps/subkey generate-node-key > .nodekey_mainnet
cat .nodekey_mainnet

see https://github.com/threefoldtech/grid_deployment/commit/1f5fd3e47703d5782bab684c06a5188419ba77f1

done


Set Lamore et al and Force et al target toxicology evidence as preclinical

Describe the bug
The safety liabilities evidence for Force et al. and Lamore et al. is missing information about the type of assays underlying the reported safety events. As reported by a Community user, this is useful to distinguish between clinical/preclinical information.
Observed behaviour
The evidence entries for these sources have a NULL value for studyType in the safetyLiabilities object.
Expected behaviour
Skimming through both publications, the nature of the evidence should be classified as preclinical. cc @inessmit

Thank you Irene for the detailed response to the user and raising a ticket for this incomplete information.
For the Lamore et al. study, it's based on human induced pluripotent stem cell-derived cardiomyocytes (Table 2 in the article), so cell-based seems the most appropriate.
I agree that for the Force et al. article, the studyType is preclinical, as it's based on mouse studies.


Add initial pt translation to network policy

Initial translation, will move from draft to PR when ready

@devlware @jailton better late then never, here it is completed and ready to be reviewed

@jcjesus @edsoncelio Fixed all the comments. Sorry for the long delay.
I'm adding the squash label to this merge, so as soon as @jcjesus approves this the commits are squashed without my iteration.
/label tide/merge-method-squash

/lgtm

/assign @jcjesus

/unassign @jailton

/lgtm

/unassign @jailton


chore: upgrade Nodejs and Yarn

Also switches to using frontend-maven-plugin to run Yarn. Not sure
about the problem encountered (error 134), I don't think it should be an
issue with this combination of versions. Can easily be reverted if
needed.

@zregvart is it really a good idea to have that yarn cache in git?

@zregvart is it really a good idea to have that yarn cache in git?

If you'd like a deterministic/offline build, that's the only option. It makes the clean build (i.e. CI) much faster as well. This is what we do on the website and main Camel repository and so far no issues. Bit more on this can be found here: https://yarnpkg.com/features/pnp

If you'd like a deterministic/offline build, that's the only option

Could you please explain the determinism aspect? Is there no way to perform yarn install so that the result is the same for everybody?

It makes the clean build (i.e. CI) much faster as well.

This sounds interesting of course. But I wonder how much faster it can get? If I delete the cache locally, running yarn install takes about 15 seconds. OTOH downloading 20+ MB via git takes some time too. So let's assume the net gain can be up to 10 seconds. Do you agree with these figures, Zoran? (Maybe I am measuring wrong processes - I am a newbie in this area.)
My main concern is that git is not very effective at maintaining binary blobs. Those zip files in the yarn cache will change over time, and the overall size of the repo will get bigger and bigger.
I must say I do not feel persuaded, that I want that for saving 10 seconds on the CI. What do others think?

I forgot to add that I rarely run the antora xref checks locally, bc . they are not included in the -Dquickly profile.

Could you please explain the determinism aspect? Is there no way to perform yarn install so that the result is the same for everybody?

In part it depends on how one specifies the versions in package.json, as NPM allows loose versions, that's the reason behind the lock files. So a npm install or a yarn install might not fetch the same version on two occasions (baring the lock file). More details on this can be found here: https://classic.yarnpkg.com/blog/2017/05/31/determinism/ and here https://npm.github.io/npm-like-im-5/npm3/non-determinism.html. Yarn's PnP was rethink of the node_modules bit and having the cache in source control is more one step along that path.
An alternative could be to have a shared Yarn cache, that would need some work on the CI (to make sure cache is available and up to date).

Could you please explain the determinism aspect? Is there no way to perform yarn install so that the result is the same for everybody?

In part it depends on how one specifies the versions in package.json, as NPM allows loose versions, that's the reason behind the lock files. So a npm install or a yarn install might not fetch the same version on two occasions (baring the lock file). More details on this can be found here: https://classic.yarnpkg.com/blog/2017/05/31/determinism/ and here https://npm.github.io/npm-like-im-5/npm3/non-determinism.html. Yarn's PnP was rethink of the node_modules bit and having the cache in source control is more one step along that path.

Isn't yarn install --immutable solving the reproducibility problem well enough? The docs says it is the replacement for --frozen-lockfile  that we used so far. yarn install --immutable takes 606ms if the .yarn folder is available and up to date. So for local development, the cache folder can just stay outside of git and yarn install --immutable invoked by the frontend plugin can care for keeping it up to date if necessary. If the dev does not delete it manually, the rebuilds should be fast. On the CI, it can be cached much like ~/.m2/repository, no?

@ppalaga yes, as I've outlined in the last sentence of the last comment I made, this can be done as you describe using the shared Yarn cache. I think the way I've done it here is better, do you have any objections to doing it this way?

I have no strong preference for how we do this, but I lean towards Peters' approach if it can reduce repo bloatage.

I'd prepare a new PR based on this one, but without adding the ZIP files to git. Would that work for you, @zregvart?


Update release info, include eBPF

@mariomac : adding to the agent field description: "The eBPF agent is not officially released yet, it is provided as a preview."

/approve


Update the demo code

app.Run()  => app.Start() in the demo code at gofr.dev .

Current code gives error.

Hey, its app.Run() only - please check the latest code.
https://github.com/gofr-dev/gofr/blob/development/examples/http-server/main.go


Chart.defaults.global.responsive setting not always works

Version 2.0.0-alpha4
I'm struggling to find the cause this problem, so I'm going to share what I do know.
First of all, I think this line:
https://github.com/nnnick/Chart.js/blob/v2.0-dev/src/core/core.js#L63
// Always bind this so that if the responsive state changes we still work
61 		var _this = this;
62 		Chart.helpers.addResizeListener(context.canvas.parentNode, function() {
63 			if (config.options.responsive) {
64 				_this.controller.resize();
65 			}
66 		});

Should be:
if (_this.config.options.responsive) {

unless I'm missing something.
I have changed it in my code and it's working but only if I have the debugger running and a breakpoint placed on line 63. If I remove the breakpoint, it stops working again.
I've been using the line sample: https://github.com/nnnick/Chart.js/blob/v2.0-dev/samples/line.html to test this and the only change I make is the one listed above (to Chart.js file) and adding:
window.onload = function() {
        Chart.defaults.global.responsive = false;

        ...
    };

in the line.html file.

Thanks @lpaluszk I tracked down 2 issues with this. The first was a race condition that could easily be reproduced in the debugger if _this.controller had not yet been created. The second was that the config is not merged with the global configuration until the Chart.controller constructor. I've solved this by building the controller before adding the resize listener and looking in the merged options stored in the controller for the responsive setting.
I tested this in the line sample by setting the canvas to a fixed width and height and then turning off the responsive global setting and removing the responsive: true from the chart settings object in the sample file. After doing so, the results were as expected.
The canvas was set up as follows for my test.
<div style="width:500px;">
    <canvas id="canvas" style="width:500px;height:250px" width="500" height="250"></canvas>
</div>

See #1526 for a fix to this.

That was quick. Thanks.
When are you planning the next release for?

Hopefully quite soon. I think all the major features for v2.0 are done.

I can't seem to get it working.
Something doesn't get merged in properly. Please see screenshot.

@lpaluszk are you still using the line sample file? If so, it has the responsive option passed in as true for the chart so you need to remove that as well.

@etimberg I am, but I have now removed responsive: true from this piece of html/js:
https://github.com/nnnick/Chart.js/blob/v2.0-dev/samples/line.html#L60-L78
and I'm still getting a responsive chart.
Here's my line.html file - http://pastebin.com/gEVemTDx

@lpaluszk I went and looked at this again. I used your line.html directly and it worked as expected :/

I've refrehed my local repo and rebuilt it with gulp. I can see all the latest commits in Chart.js file.
The chart is always responsive. I tried changing the option from responsive: true to responsive: false but this doesn't do anything either.
Can you share your working line.html and Chart.js?

I created a JSFiddle with what I have on my machine. The responsive setting should only affect resizes and when I resize the page, the chart does not resize. My test was on Chrome 46 on Mac OS X 10.10.5

Your JSFiddle is working as expected. I copied all JS from the fiddle into my line example and the chart was still responsive.
But if I use HTML from your fiddle I'm getting a non-responsive chart. The difference is with canvas size style:
https://github.com/nnnick/Chart.js/blob/v2.0-dev/samples/line.html#L17
<canvas id="canvas" style="width:100%;height:100%"></canvas>

vs
<canvas id="canvas" style="width:500px;height:250" width="500" height="250"></canvas>

The former always causes the chart to be responsive.
Testing on Windows 10, IE 11, latest Chrome and latest FF.

ah, ok, I see what's happening. Because the canvas is styled with a percentage width and height, resizing the window changes the canvas CSS size. However, the chart does not actually re-render for this new size. You can see this by making the window very small, refreshing the page and then increasing the size of the page. The chart appears very pixelated because it does never re-renders for the new size.
I think your best bet is set the canvas to a fixed size if you don't want your chart to change size as your window size changes.

I actually want my canvas to be responsive. I came across this behaviour and thought it was a bug so I reported it.
It does indeed work as expected now.
Thanks

:smile:  There was a subtle bug with the global setting merge there, so great catch on that!


Profile Creation Issues

Describe the bug
Long story short, whenever I try and create a new profile for a pet and/or companion, or even try and add character traits below the basic info such as age, race, etc, I get half a second worth of lag and nothing happens. I've tried mass-deleting everything TRP related and reinstalling newer versions several times but nothing seems to work regardless.
To Reproduce
Steps to reproduce the behaviour:

Go to 'About'
Click on 'Add Trait'
Scroll down to '....'
See error

Expected behaviour
After restoring everything, the expected result would be to be able to produce pet or mount profiles again. Still no result.
Screenshots or error message
If applicable, add screenshots to help explain your problem.

If you don't see any errors, make sure that error reporting is enabled (/console scriptErrors 1) or install the BugSack add-on.

Version of the add-on:
Total RP 3 2.3.0
Did you try having Total RP 3 as the only enabled addon and everything else (especially something like ElvUI) disabled?
No
Additional context
If your issue is related to profile data, please provide your data file.

It is the totalRP3.lua file (not the .lua.bak files!) located in your World of Warcraft/_retail_/WTF/Account/ACCOUNT_NAME/SavedVariables folder (replace ACCOUNT_NAME with your actual account name, visible on the Battle.net launcher, usually WoW1 for accounts created after the introduction of Battle.net; replace _retail_ with _classic_ if you are using Total RP 3: Classic). You can just drag and drop it here, no need for a third party service.

Did you try having Total RP 3 as the only enabled addon and everything else (especially something like ElvUI) disabled?
No

Try that first, outdated addons using one of the libraries we use (like CrossRP) cause that issue.

Did it to no avail. Not to worry though, clearing back-up files and re-installing the game twice fixed it. Thanks for the assist though!


Please support Astro 4

Thank you

Hey @colinrobinsonuib ! It should be supported, I believe. Are there any specific Astro 4 issues you've experienced with the package?

Hi,

It can't be installed because of the dependency "astro": "^3.6.4" so if you could bump the major version number, that would be good.

Even if I force the install by doing

"overrides": {
    "relatinator": {
      "astro": "$astro"
    }
  }

I get the error on build:
[vite] Error when evaluating SSR module C:\project\astro.config.mjs: failed to import "/node_modules/astro-relatinator/integration.ts" |- Error: Failed to resolve entry for package "relatinator". The package may have incorrect main/module/exports specified in its package.json.

@colinrobinsonuib Should be good now, latest in matter (fixed by https://github.com/DBozhinovski/relatinator/commit/e37fc37edae5324f21cf886ef6fa2e62ddbcbca9)

Also published to NPM now.


Use data and lock

migrated from NSMulatbleData to Data
marked APIManager as final class
added an NSLock instance to the APIManager class to synchronize access to shared resources. Swift dictionary is not thread-safe and accessing it from singleton instance (Flagsmith.shared) can lead to race conditions.

After two weeks of observation, I can state that it fixes our crashes on production. Please take a look at it, since we want to drop an internal fork. cc @dabeeeenster

We're going to write some unit tests around this first I think!

I think this can be closed as superseded by https://github.com/Flagsmith/flagsmith-ios-client/pull/45


Edit metadata redesign

References

Fixes #1722
Fixes #1894

Description
This PR redesigns the edit-metadata page for items (and adds support for other DSpaceObjects), solving several issues (see references)
Instructions for Reviewers
Technical changes:

Added a new module called DsoSharedModule, which can be imported wherever an edit metadata page is required for any DSpaceObject (and possible future shared components)
Within this module, the main component for metadata edit is DsoEditMetadataComponent (ds-themed-dso-edit-metadata to support themes)
DsoEditMetadataForm is a helper class dynamically storing information about the object's metadata and its changes made to them. It's also responsible for turning all the metadata changes into an array of Operations which can be sent to the REST API to update our object

Functionality changes:

Metadata values are grouped by their field
Metadata values are draggable within their field to change their place property
Adding a new metadata value will automatically add it to the bottom of their respective field
Metadata fields are sorted alphabetically

Testing guidelines:

Edit an item and visit the "edit metadata" tab
Click the add button to start adding a new value
You should be able to search through all metadata fields using the field input
Clicking the "confirm" button to the right of the value should add the new value at the bottom of their respective field and remain marked green
Clicking the "confirm" button whilst an invalid metadata field is filled in, the user should get a warning underneath the input and the value isn't added
Clicking "undo" (or "delete" on a confirmed value) should simply remove the new value
Editing an existing metadata value should mark the value orange
Marking an existing metadata value as "deleted" should mark the value red
Moving a metadata value within its field should mark the values orange which's "place" properties are changed by the move
After a move took place, the user should see an "undo" button below the respective metadata field and clicking it should undo any move changes within this field
Clicking "discard" at the top or bottom of the table after any changes were made should get rid of these changes and revert the metadata table to its original state
The "reinstate" button should be available after a discard took place and clicking it should bring back the changes that were previously discarded
Clicking "save" should send a PATCH request to the REST api containing the changes made by the user (note: move changes are calculated dynamically and the operations might not reflect the order the user moved values in, but should have the same result regardless)
A success notification should display once the object is successfully updated
Editing a collection's item template should work the exact same as editing an item

Checklist
This checklist provides a reminder of what we are going to look for when reviewing your PR. You need not complete this checklist prior to creating your PR (draft PRs are always welcome). If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!

[x] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & specs/tests), or I have provided reasons as to why that's not possible.
[x] My PR passes ESLint validation using yarn lint
[x] My PR doesn't introduce circular dependencies (verified via yarn check-circ-deps)
[x] My PR includes TypeDoc comments for all new (or modified) public methods and classes. It also includes TypeDoc for large or complex private methods.
[x] My PR passes all specs/tests and includes new/updated specs or tests based on the Code Testing Guide.
[x] If my PR includes new libraries/dependencies (in package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
[x] If my PR includes new features or configurations, I've provided basic technical documentation in the PR itself.
[x] If my PR fixes an issue ticket, I've linked them together.

@atarix83 Thank you for the feedback!
Those are some valid concerns, so I've fixed the dragging issue and increased the amount of rows on textareas to 5.

Second, while the features here are great. This seems to have lost all the accessibility tags/elements that we had in the old "Edit Item" screen. Most specifically, we stopped using  (which is problem for accessibility, see inline notes below), and all buttons are missing title attributes. I think the buttons are an easy fix. Not as sure about the  elements, but it'd be nice to move back to them if at all possible.

@tdonohue We tried to do this using tables first, but we had to move away from them (and do all this work to refactor it to look the same using flex-box). When using tables we needed to use rowspan on the field name cells, which means the first row for a metadata field would have one cell more than the rows below. Not only did that make the html significantly more complicated, it led to all sorts of issues with drag-to-reorder, as you would be dragging rows of different sizes and outside of the context of the table (while dragging) rows wouldn't completely change their shape. So we can't go back to using <table> if we want to keep the grouping  by field, and drag to reorder
Instead I propose we add role  and aria attributes to the current design to make it as accessible as possible: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Roles/table_role

@artlowel : Just to verify... Yes, I'm ok with using the aria-role attributes as a "workaround" to the fact that <table>s are hard to implement for this page.  I just want to be sure we are making this page as accessible as possible.

@tdonohue Those roles and aria attributes have been added


Initialize model from the start

Currently, the model isn't initialized until a dataset is initialized. By default, the model should do the random blob initialization that it currently does for NeRF synthetic data as soon as the server is started. If a SfM dataset is loaded, the UI should ask if the user would like to load the SfM points.

Adjusted model loading code to initialize the model to the random blobs it uses for Nerf synthetic training.


Bible.com

Thanks for this. Very handy, however it seems to be getting stuck on bible.com. It just sits there at 0%

It's actually working now, my mistake. Thanks.


convert non-normative note into constraint

This non-normative note included RFC2119 keywords, making its
status as a note ambiguous. Per our previous discussion, json-api
should not disallow implementors from including hasOne fkeys as
attributes, but it should require that any relations which are
provided must be represented under the links object.

:+1:

@tkellen There's one occurrence of *SHOULD NOT* instead of **SHOULD NOT**.

Thanks for catching that @bintoro, will push a fix after we land #417.


Replace react-icons dependency with official Font Awesome React component

Feature
Why is this feature being added?
We are currently using the react-icons to include Font Awesome icons in the front-end.  That library is no longer being maintained, and also is missing some of the newer icons that are included in Font Awesome 5.
Font Awesome now provides an official React component that can replace this: https://github.com/FortAwesome/react-fontawesome . This will allow us to have access to all Font Awesome icons, and also allow use the official component which continues to be maintained.
The official FA React component uses a simple declarative syntax:
<FontAwesomeIcon icon="coffee"/>
What should your feature do?

Update dependencies in package.json
Replace implementations of the old react-icons dependency with the new Font Awesome React component.

@jjhampton Is this issue still open?

@dave5801 Yes, however it's currently assigned and has a PR that has been submitted and is waiting in the backlog for review/approval.


Expose serializer.ts methods in public API

Is your feature request related to a problem? Please describe.
My team and I have written a Cloud Function that listens to Cloud Firestore google.cloud.firestore.document.v1.written events, serializes them as JSON objects, and publishes them to a Pub/Sub topic. We followed the latest example which encourages the use of CloudEvent and protobuf. The problem is... there isn't much in the way of useful documentation, code, or libraries for converting that decoded cloud event to a JSON representation (e.g., without things like "isActive": {"booleanValue": true }, "datePurchased":{"seconds":"1717145220","nanos":90913000}, "middleName":"NULL_VALUE", etc.).
Describe the solution you'd like
Make decodeValue available in the public API.
Describe alternatives you've considered

Write my own decoder that largely copies and pastes from serializer.ts
Switch away from CloudEvent in favour of more classic Firestore Functions similar to what is done in the firestore-bigquery-export extension

Additional context
Ideally there'd be a Firebase extension similar to firestore-bigquery-export that routes messages to Pub/Sub rather than BigQuery. Or even an EventArc Destination that is just a Pub/Sub topic?

Thank you for your request and I apologize for the difficulty you are experiencing. After talking with the team, we have decided not to expose the functionality from serializer.ts as part of the public API. The class has many transitive dependencies on other internal code which we are not prepared to expose and maintain.

@dconeybe okay, alternatively, can you or the team provide guidance or an example of how to translate a CloudEvent document back to a JSON representation? (or reassurance that such a mechanism is coming eventually?)

One option you could consider is to use https://github.com/firebase/firebase-functions, e.g.
import {
  onDocumentWritten,
  Change,
  FirestoreEvent
} from "firebase-functions/v2/firestore";

exports.myfunction = onDocumentWritten("users/marie", (event) => {
     // Get an object with the current document values.
    // If the document does not exist, it was deleted
    const newValues =  event.data.after.data();

    // Get an object with the previous document values
    const previousValues =  event.data.before.data();
})

You don't necessarily have to use the Firebase CLI to deploy your functions written with Firebase Functions SDK, though you may need to add yet another dependency on firebase-admin(https://firebase.google.com/docs/admin/setup#add-sdk) since Firebase Functions SDK relies on some of its feature to construct a write-able Firestore snapshot.


Demo is missing, please resolve

Demo of your template is missing, please resolve so we can see how it would look. Thank you.

http://www.gdgmanagua.org/jekyll-mdl site cannot be reache

Fixed, thanks


Getting error while pushing logs

Getting the below error and I verified "google-cloud-logging" is latest version. can you tell me what is wrong. Tried running this code from Google cloud shell.. Dataflow is up and running but de-identify log work is not happening.
2022-02-14 11:52:03.561 ISTError message from worker: generic::unknown: Traceback (most recent call last): File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs File "/home/parthepan/deidentify_logs/deidentify_logs.py", line 85, in process AttributeError: module 'google.cloud' has no attribute 'logging' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute response = task() File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in  lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 581, in do_instruction getattr(request, request_type), request.instruction_id) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 996, in process_bundle element.data) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 346, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 915, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 919, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1265, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1361, in apache_beam.runners.common._OutputProcessor.process_outputs File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1265, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1361, in apache_beam.runners.common._OutputProcessor.process_outputs File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs File "/home/parthepan/deidentify_logs/deidentify_logs.py", line 85, in process AttributeError: module 'google.cloud' has no attribute 'logging' [while running 'Send to Cloud Logging-ptransform-60']

thank you for letting know about the problem. i will look into it shortly.
it is possible that the problem is related to the fact that the latest version of the google-cloud-logging library for Python is >= 3.0.0 and contains breaking changes compared to versions < 3.0.0
As a temporary workaround, please check if the version of the logging library is indeed >= 3.0.0 and try to downgrade it to the previous (2.7.0)

Hi @minherz this is Ibrahim working with @sparthe123 and we're from same organisation.
It  looks we're facing the same issue again after downgrading the logging library to 2.7.0  and 2.6.0 respectively. Please shed some thing which we can try it out, also found there was a commit made to the same file recently just wondering if that could be case of this issues. Thanks again..

The last change was to fix a problem with the DLP package reference that raised after the recent changes.
I have a limited capacity this week, but will do it as soon as possible.

i apologies for the delay.
yes, i continue working on it. the error was due to the fact that the standard Dataflow image does not include google.cloud.logging package. I've added a custom image that includes it and now validate the code and scripts to make it work. to save the time i will generate a publicly available custom image as well.
i estimate that these changes will be rolled in next week.

Hi.
So  I've created the custom image and pushed to GCR and  have used that  image under /pipiline/install.sh looks it's throwing Frameworks/Python.framework/Versions/3.9/bin/python3: No module named deidentify_logs error..
It couldn't able to find deidentify_logs module.
I've also tried  with your GCR image as well showing the same. can you help on this pls to get sorted

@minherz also looks the custom_image is not running.. when tried calling locally.. where able to find this https://stackoverflow.com/questions/69195731/dockerized-apache-beam-returns-no-id-provided Thanks for all you do.

Yeah, i had few left overs after pushing the change. I added more info into README about running the PoC and its limitations. I also fixed the pipeline/install.sh so it will use the renamed module.
Let me know if there are more questions.
P.S. I will edit the post on Medium to align it with the recent fixes during the next week.

When you said "running" what do you mean? I was notified (see also #5) that install.sh was not modify to reflect on the argument renaming. I've fixed that. It was my mistake. I did not test the install.sh script. I will run the full test later this week.

Hi @minherz was referring to the same on the 3rd issue.. #5
Now while running the script happen to see the following error.
INFO:root:starting the pipeline to read from:projects/None/subscriptions/log_ingestion_subscr Traceback (most recent call last):   File "/Users/ibrahim.munir/Downloads/WP - Standard/ANCT/dlpfinding/deidentify_logs/deidentify_logs/pipeline/deidentify.py", line 214, in <module>     run()   File "/Users/ibrahim.munir/Downloads/WP - Standard/ANCT/dlpfinding/deidentify_logs/deidentify_logs/pipeline/deidentify.py", line 195, in run     | 'Read exports from Pub/Sub sink' >> ReadFromPubSub(subscription=input_subscription).with_output_types(bytes)   File "/Users/ibrahim.munir/Library/Python/3.9/lib/python/site-packages/apache_beam/io/gcp/pubsub.py", line 223, in __init__     self._source = _PubSubSource(   File "/Users/ibrahim.munir/Library/Python/3.9/lib/python/site-packages/apache_beam/io/gcp/pubsub.py", line 429, in __init__     self.project, self.subscription_name = parse_subscription(subscription)   File "/Users/ibrahim.munir/Library/Python/3.9/lib/python/site-packages/apache_beam/io/gcp/pubsub.py", line 390, in parse_subscription     raise ValueError('Invalid PubSub project name: %r.' % project) ValueError: Invalid PubSub project name: 'None'.
It looks that for projects/None/subscriptions/log_ingestion_subscr --> it's not capturing the project ID, thought of passing it manually but didn't worked either. Can you help checking this once for us pls. Thanks again.

It is strange. It can happen only if you run it locally and the GOOGLE_CLOUD_PROJECT environment variable is not defined.
I will add a proper validation to the code to ensure that the project is always provided :+1:

Hi @minherz I've ran the  script under cloud console  and were able to  see de-identified logs, but it only prints  on  the logging and while trying to  see  under command line it  throws an error -->> WARNING: The following filter keys were not present in any resource : logs Listed 0 items.
Looks  it's  only showing up for the Phone number  and not email, thought of sharing this as well.
Also were able to see errors happening  under dataflow console attaching here
2022-03-09 21:32:37.853 ISTError message from worker: generic::unknown: Traceback (most recent call last): File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs File "/home/ibrahim_munir/deidentify_logs/pipeline/deidentify.py", line 141, in process UnboundLocalError: local variable 'gcp_logging_client' referenced before assignment During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute response = task() File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle input_op_by_transform_id[element.transform_id].process_encoded( File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 346, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 915, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 919, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1265, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1361, in apache_beam.runners.common._OutputProcessor.process_outputs File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1265, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1361, in apache_beam.runners.common._OutputProcessor.process_outputs File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs File "/home/ibrahim_munir/deidentify_logs/pipeline/deidentify.py", line 141, in process UnboundLocalError: local variable 'gcp_logging_client' referenced before assignment [while running 'Send to Cloud Logging-ptransform-60']
Thanks again @minherz.

Nvm  with reg. Phone number looks the new code  base does only have phone  number not anymore maybe we  should update in our end with reg. info_types  to see if that works..
`# identify payloads
INSPECT_CONFIG = {
"info_types": [
{"name": "US_SOCIAL_SECURITY_NUMBER" }
]
}
DEIDENTIFY_CONFIG = {
"info_type_transformations": {
"transformations": [
{
"primitive_transformation": {
"character_mask_config": {
"masking_character": '#'
}
}
}
]
}
}
`

@ibrahimejaz I have removed masking the phone number. For a proof of concept the SSN masking should be sufficient.
Please, let me know if you find it hard to extend. While I do not plan to convert this PoC to a reference solution for now, I will be glad to assist you.
Regarding the working script, I will check how it works end-to-end in the next couple of weeks.

@minherz were able to see this error many times.
We can see it's running but the no-pii-logs is not printing in cloud logging.. It says the count but prints only one time.. attaching logs for reference
2022-03-18 14:46:42.004 ISTError message from worker: generic::unknown: Traceback (most recent call last): File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs File "/home/ibrahim_munir/deidentify_logs/pipeline/deidentify.py", line 142, in process UnboundLocalError: local variable 'gcp_logging_client' referenced before assignment During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute response = task() File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction return getattr(self, request_type)( File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle input_op_by_transform_id[element.transform_id].process_encoded( File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 346, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 915, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 919, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1265, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1361, in apache_beam.runners.common._OutputProcessor.process_outputs File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1265, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1361, in apache_beam.runners.common._OutputProcessor.process_outputs File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs File "/home/ibrahim_munir/deidentify_logs/pipeline/deidentify.py", line 142, in process UnboundLocalError: local variable 'gcp_logging_client' referenced before assignment [while running 'Send to Cloud Logging-ptransform-61']
Also attaching screenshot
https://share.anysnap.app/fyw3gh6UEKRH
Per the screen shot you may see it's actually printing no.of times it got scanned but in the logs it does shows only one time..
means it's de-identifying fine but while printing it's not printing the actual de-identified logs
Can you please shed us your thoughts on the same.

@minherz
Also can you shed us some light on how we can apply the same masking for gae_app logs completely.. or any inisights on implementing the same.. Thanks again.

I am checking the problems with installation script when launching it first time on a fresh project. I will update the README to reflect constraints if I find any.
About your question: You can control the logs de-identification by configuring the route rules that stream ingested logs into PubSub. The blog post describes how to do it for log entries with a particular log name but you can easily modify it to include all application logs with the AppEngine resource type or to narrow it to include the service name and version.
Be mindful that this proof of concept does not exclude ingested logs resulting in keeping the original log entry in addition to the de-identified. It was made to simplify comparison.
Let me know if you need assistance in understanding how to exclude specific log entries from being ingested.


remove configmap/status resource from leader-election-role

fixes #44

This looks good to me. I've tested the same change and everything appears to run without issue.


new repowriteproxy to funnel writes that need crud notifications thru

Coverage increased (+2.0%) to 73.038% when pulling 9044d67d7734b771f22be89b4f29a0521b81fdf4 on crud_notification_funnel into 8528e0077a847f44421bffedd52adc0824074ad0 on master.

Coverage increased (+2.0%) to 73.038% when pulling 9044d67d7734b771f22be89b4f29a0521b81fdf4 on crud_notification_funnel into 8528e0077a847f44421bffedd52adc0824074ad0 on master.

Coverage increased (+2.0%) to 73.038% when pulling 9044d67d7734b771f22be89b4f29a0521b81fdf4 on crud_notification_funnel into 8528e0077a847f44421bffedd52adc0824074ad0 on master.

Coverage increased (+1.6%) to 72.636% when pulling 9044d67d7734b771f22be89b4f29a0521b81fdf4 on crud_notification_funnel into 8528e0077a847f44421bffedd52adc0824074ad0 on master.


Welcome header

Fix #24
Comments:

Finished styling of header tab effect

As a programmer,
I want to adapt my tab color,
so I can have tab effects.

Very good job Jonas! One question, have you tried the color of the search icon as border of the active tab? Or you take the opacity of the active tab away? Thought that might be nice as well.


ingress removal

Pull Request Review
Checklist

[x] Issue: #88
[x] List of related PRs. : None
[x] Project builds without any errors.
[x] Unit Tests updated (or created where needed) and all pass.
[x] You have ran manual functional “smoke” test (does product as a whole still work?).
[x] User Story meets the acceptance criteria - and passes. acceptance criteria should be understood at outset.
[x] There is a PR for update of internal documentation for the affected Project(s) - README etc.
[x]  Description of the changes:  #88

Another issue : https://github.com/IBM/openstack-vim-driver/issues/91

Raised a new PR (#92)as the Id got changed for Manoj Natarajan.


"®" symbol in pkg-config file breaks downstream build tools

Hi there! I've been digging into https://github.com/haskell/cabal/issues/9608, which is caused by being unable to parse output from pkg-config --list-all when libvpl is installed.
This is because a "®" symbol was added to the Name and Description properties of vpl.pc when this project was renamed from "oneVPL" to "libvpl". See the file https://github.com/intel/libvpl/blob/master/libvpl/pkgconfig/vpl.pc.in in commit https://github.com/intel/libvpl/commit/44df6e08cc836ed65725ed344e7ec9ded5193874.
This results in an invalid pkg-config file. Note that the pkg-config manpages (link) define:

Properties are set using RFC822-style stanzas which consist of a keyword, followed by a colon (:) and then the value the property should be set to.

And RFC-822 defines property field bodies to be ASCII characters only, and "®" is not an ASCII character.
Can you please remove this ISO/IEC 8859-1 symbol from the .pc output files? Thanks!

@elldritch Thanks for the report! We will make sure is fixed in the next release!

@mav-intel Thank you! When do you expect the next release to be? This is breaking my builds today, so I am figuring out whether to work around it by temporarily downgrading or whether to temporarily patch it myself.

Fixed by #121


Whitelist validation for RemoteImageService: too permissive??

Prerequisites

[x] I have written a descriptive issue title
[x] I have verified that I am running the latest version of ImageProcessor
[x] I have verified if the problem exist in both DEBUG and RELEASE mode
[x] I have searched open and closed issues to ensure it has not already been reported

Description
For my purposes I think that there may be a small issue with the whitelist validation in RemoteImageService, in that it's too permissive with sub-domains.
See IsValidRequest method: https://github.com/JimBobSquarePants/ImageProcessor/blob/develop/src/ImageProcessor.Web/Services/RemoteImageService.cs#L79
I was already implementing a custom version, so used the following variation:
            // Check the url is from a whitelisted location.
            var url = new Uri(path);
            var upper = url.Host.ToUpperInvariant();

            // Check for root or sub domain.
            foreach (var uri in ImageProcessingWhitelist)
            {
                var compareUri = uri;
                if (!compareUri.IsAbsoluteUri)
                {
                    compareUri = new Uri("http://" + compareUri.ToString().TrimStart('.', '/'));
                }

                if (upper.StartsWith(compareUri.Host.ToUpperInvariant())
                    || upper.EndsWith($".{compareUri.Host.ToUpperInvariant()}"))
                {
                    return true;
                }
            }

            return false;

I recognise that just blindly assuming that what's in the whitelist is a complete (sub-)domain and pre-pending a period may not be what everyone actually needs; it just suited my purpose.
Steps to Reproduce

Add a sub-domain to the whitelist configuration, e.g. sub.domain.com
Request an image that is hosted at a different - but similar - subdomain, e.g. invalid-sub.domain.com
The image will be served, even though it's on a different sub-domain

System Configuration
Local debugging / unit testing

ImageProcessor version: 2.5.4
ImageProcessor.Web version (if applicable): 4.8.4
Other ImageProcessor packages and versions:
-- ImageProcessor.Web.Config v2.3.0
Environment (Operating system, version and so on): Windows 10
.NET Framework version: 4.6.2
Additional information:

Hi @natwallbank
The whitelist is specifically designed to be as permissive as you make it. the more detail you put in, the more restrictive. It needs to allow developers to be as loose as they need to also.
Cheers
James

Hi @JimBobSquarePants.  Thanks for the swift reply and thanks for the awesome library!
No problem, it was just something that surprised me after I'd configured a load of unit tests and one of them failed, testing a different sub-domain.  I thought it was worth mentioning but didn't want to presume that it was not by design!
Cheers,
Nat


Adds an Alias action

Signed-off-by: Drew Baugher<EMAIL_ADDRESS>Issue #, if available:
https://github.com/opensearch-project/index-management/issues/35
Description of changes:
Adds support for an alias action in ISM.
Gives complete support for all update alias API options by reusing the alias actions parser internally and explicitly not defining the mappings in our config index.
Coming from the GitHub feature request, the use cases people asked for:

One use case we have is for indices older than a certain age, we allocate them to a cold node, shrink them, delete the original and create an alias with the original name that points to the shrink index. This keeps our shard count down while not breaking queries for older data.
1.1. As of right now I don't believe this would be doable because we do not know the name of the source index that was used to shrink to this target index. To do so we need to update core to store this information in the IndexMetadata when the shrink operation is performed. This theoretically should be doable though once we release the Shrink action.. in a roundabout kind of way. Essentially once ISM shrinks foo-index to foo-index-shrink there will be two jobs running. One for foo-index and one for foo-index-shrink. If we knew what the source index of the shrink was on foo-index-shrink then foo-index-shrink could call the update alias API and delete the foo-index and add that as an alias to itself in a single API call I believe. Since that's not possible for now.. then going the other way the foo-index after it successfully executes the Shrink action should be able to do the same thing, it can call the update alias API and delete itself while also setting an alias on the foo-index-shrink equal to it's own index name.
1.2PUT foo-index
PUT foo-index-shrink
POST /_aliases
{
  "actions" : [
    { "remove_index": { "index": "foo-index" } },
    { "add" : { "index" : "foo-index-shrink", "alias" : "foo-index" } }
  ]
}

My use case is when an index foo is shrunk, it becomes shrink-foo. An alias action would allow the new shrunk index to also be aliased to foo so applications don't know the difference. When we moved from x-pack and ILM to ISM, the aliasing had to be handled with custom code outside ES. I'd be just as happy with the alias being an optional setting on the shrink operation. Happy to share details, but what we're doing is pretty basic.
2.1 I'm assuming this is similar to (1) because I don't believe you can add an alias that is a concrete index. So you would need to delete the foo index in this example. So if it's the same as (1) then yes, there would be a way once we support Shrink action.
my use case is around the shrinking of indices as well. But in general I think it would be great to support adding an alias, especially if we get to the stage where we can roll up indices into single indices then alias them with the ones that were rolled up.
3.1 Should be similar to previous cases where the job that performs the rollup would have to then go and update the target index to have aliases of the source index patterns. The downside being I don't believe this specific policy would be that reusable.. and it wouldn't be possible to go the other way, i.e. a target rollup index comes into being and starts being managed and then figures out which indices it was rolled up from (we might actually store that information in the meta mappings of the rollup target index, so it could be doable in the future). Or if the source index has a wildcard pattern like log* then when updating the alias for the target index.. we can't use that and would have to resolve to concrete indices which could be old or become stale.
My use case is, If we have an alias for an index with rollover ILM policy, can we remove it from alias after some predefined days, let's say 30 days? I know there is a way to delete an index once its old, but in our case we do not want to delete but want the index to be removed from alias and have the alias switch and point to indices whose age is less than 30 days.
4.1. This should be easily doable with the Alias action, just have a state that gets transitioned into after the index is 30 days old (or 30 days rollover age) and then remove the alias.
My use case is: we provide search_alias to search our dataset. We publish new version of dataset every week as a new index (e.g. dataset-yyyy-MM-dd. We want the search_alias always pointing to the latest published dataset. The process is: ingest new data to a new index, once ingestion finish, re-pointing search_alias to the new index, remove old index from the search_alias
5.1 Should be possible, the newly created index can remove the alias from all search-index-* indices and then add the alias it itself in a single call. The main thing would be knowing when ingestion finished for the new index.. which you could just use some absolute time using the transition condition if you know it only takes x amount of time.

CheckList:

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #304 (ab81231) into main (4985a18) will decrease coverage by 0.15%.
The diff coverage is 67.53%.

@@             Coverage Diff              @@
##               main     #304      +/-   ##
============================================
- Coverage     75.75%   75.60%   -0.16%
- Complexity     2008     2016       +8
============================================
  Files           253      256       +3
  Lines         11466    11543      +77
  Branches       1815     1823       +8
============================================
+ Hits           8686     8727      +41
- Misses         1753     1775      +22
- Partials       1027     1041      +14

Impacted Files
Coverage Δ

...ndexstatemanagement/step/alias/AttemptAliasStep.kt
64.44% <64.44%> (ø)

...t/indexstatemanagement/action/AliasActionParser.kt
64.70% <64.70%> (ø)

...agement/indexstatemanagement/action/AliasAction.kt
78.57% <78.57%> (ø)

...anagement/indexstatemanagement/ISMActionsParser.kt
80.32% <100.00%> (+0.32%)
:arrow_up:

...arch/indexmanagement/rollup/RollupSearchService.kt
55.55% <0.00%> (-5.56%)
:arrow_down:

.../action/explain/TransportExplainTransformAction.kt
67.04% <0.00%> (-2.28%)
:arrow_down:

...rch/indexmanagement/rollup/model/RollupMetadata.kt
82.73% <0.00%> (-1.79%)
:arrow_down:

...xmanagement/rollup/actionfilter/FieldCapsFilter.kt
72.94% <0.00%> (-1.18%)
:arrow_down:

...management/indexstatemanagement/MetadataService.kt
56.55% <0.00%> (-0.82%)
:arrow_down:

...earch/indexmanagement/transform/TransformRunner.kt
85.71% <0.00%> (-0.72%)
:arrow_down:

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4985a18...ab81231. Read the comment docs.


Drop database with Lucene index complains

OrientDB Version, operating system, or hardware.

v2.0 SNAPSHOT[ ] - .18[ ] .17[ ] .16[ ] .15[ ] .14[ ] .13[ ] .12[ ] .11[ ] .10[ ] .9[ ] .8[ ] .7[ ] .6[ ] .5[ ] .4[ ] .3[ ] .2[ ] .1[ ] .0[ ]
v2.1 SNAPSHOT[ ] - .16[ ] .15[ ] .14[ ] .13[ ] .12[ ] .11[ ] .10[ ] .9[ ] .8[ ] .7[ ] .6[ ] .5[ ] .4[ ] .3[ ] .2[ ] .1[ ] .0[ ]
v2.2 SNAPSHOT[12] - .rc1[ ] .beta2[ ] .beta1[ ]

Operating System

[x] Linux
[ ] MacOSX
[ ] Windows
[ ] Other Unix
[ ] Other, name?

Expected behavior and actual behavior
One would expect database to be dropped with no complaints.
2016-10-31 13:59:33:534 SEVER {db=pAB} Cannot delete storage directory with path /srv/OrientDB2.2.x/pAB because directory is not empty. Files: [/srv/OrientDB2.2.x/pAB/luceneIndexes] [OLocalPaginatedStorage]
Steps to reproduce the problem
Install Lucene Spatial, wget  http://central.maven.org/maven2/com/orientechnologies...
Add records in class containing property with EMBEDDED OPoint.
Drop database from Studio.
An empty directory is left: /srv/OrientDB2.2.x/pAB/luceneIndexes/

@austx
Fixed it will be available in 2.2.13
Thanks


bintray: stricter checking of upload results

Parse the JSON output of the upload and publish steps to ensure that a "success" message is always seen on upload, and that the number of files published is the same as the number of files we wanted to publish as based on the bottle JSON description.
See e.g. https://github.com/Homebrew/homebrew-core/runs/755685545?check_suite_focus=true#step:8:998 for where this bit us. The upload step failed, but curl still returned a success code rather than an e.g. 409 Conflict like we usually see.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew tests with your changes locally?

Thanks for the quick review! Merging this so future Bintray issues don't slip through our CI.