ImportError: cannot import name 'hybrid' from 'tink' (unknown location)

Hi, I used PyPi to install Tink from test.pypi.org. And then I used Python 3 to try out Python code sample in the repo. Below is the issue I got:
>>> import tink
>>> from tink import hybrid
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'hybrid' from 'tink' (unknown location)
>>> from tink import signature
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'signature' from 'tink' (unknown location)
>>> from tink import aead
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'aead' from 'tink' (unknown location)

Could did I miss? Thank you!

Which OS and Python version is this on?

@kste I have reinstalled python 3.7 on MacOS, uninstalled Tink, rebuild and reinstalled Tink. Now I have this error:
Python 3.7.5 (default, Oct 22 2019, 10:35:10)
[Clang 10.0.1 (clang-1<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tink
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/znifly/tink/python/tink/__init__.py", line 21, in <module>
    from tink import _keyset_handle
  File "/Users/znifly/tink/python/tink/_keyset_handle.py", line 25, in <module>
    from google.protobuf import message
ModuleNotFoundError: No module named 'google'

I do have protoc installed on my Mac.
protoc --version
libprotoc 3.12.3

Did pip install the protobuf package (pip3 freeze)? This should have happened automatically when running pip3 install . in the Tink folder, as it's part of the requirements.txt.

@kste Yes :) Here is output of pip3 freeze and I have protobuf installed.
pip3 freeze
absl-py==0.2.1
protobuf==3.12.2
six==1.10.0
tink==1.4.0rc2

I managed to make it work with a virtual env, rebuild with bazel and install with pip3.


Unsupported in CLI

[x]  Site search box
[x] Bread Crumb

Site search box need to be done using API method only


Custom rule function FieldLevel.FieldName() return empty string on Validate.ValidateMap()

[x] I have looked at the documentation here first?
[x] I have looked at the examples provided that may showcase my question here?

Package version eg. v9, v10:
v10
Issue, Question or Enhancement:
Custom rule FieldLevel.FieldName() return empty string on Validate.ValidateMap()
Code sample, to showcase or reproduce:
main.go
package main

import (
	"fmt"
	"reflect"

	"github.com/go-playground/validator/v10"
)

func main() {
	validate := validator.New()
	validate.RegisterTagNameFunc(func(field reflect.StructField) string {
		fmt.Println("TagNameFunc")
		return "test"
	})
	validate.RegisterValidation("presented", func(fl validator.FieldLevel) bool {
		fmt.Println("presented")
		fmt.Println("fl.FieldName()", fl.FieldName())
		return false
	}, true)
	data := map[string]interface{}{
		"field": "nil",
	}
	rule := map[string]interface{}{
		"field": "presented",
	}
	result := validate.ValidateMap(data, rule)
	fmt.Println("result", result)
	err := result["field"]
	fmt.Println(reflect.TypeOf(err))
	fmt.Println(err)
}

output:
presented
fl.FieldName()
result map[field:Key: '' Error:Field validation for '' failed on the 'presented' tag]
validator.ValidationErrors
Key: '' Error:Field validation for '' failed on the 'presented' tag

Expected
FieldLevel.FieldName() will return string "field" instead of empty string "".

It's also happened to me.
In my case it's happened when I'm trying to validate map using ValidateMap and then trying to cast the result into validation.ValidationErrors.
The error.Field() and error.StructField() always returning empty string.
Probably this is a bug when validating a map, the key (field name) is not returned back.

I am also having this issue: I use ValidateMap but cannot get the field name


SW3 - Semantic block columns

A.Liv

Yeah I get that. Hmm... ponder ponder.
Any chance that each column can be a contiguous block of text, and other parallell columns can auto-update with changes made in any of the other?
I realize that'll be way trickier to set up data structure wise probably... 😅
But honestly, that'd be cool to me. It'd keep the advantage of copy/pasting longer blocks of stuff, say from scala files. But also have the advantage of separating different types of semantics.

frostburn

That might cover most use cases. I just want SonicWeave to be a full programming language for manipulating absolute and relative frequency quantities so it's too Big-and-Cool ™ to fit into neat little columns like that. I need to think about it...

frostburn

If I can make this work and extend it both ways with virtual (non-editable) rows, it could replace the second column a.k.a. tuning table of SW2 freeing up a ton of screen space.

A.Liv

Good point yeah!
That's my main personal criticism of version 2 tbh. A lot of space on the main page is taken up by that table, and personally I haven't felt like I need to look at that very often (though of course others might feel differently).
Yeah honestly, the more I think about it the more this idea seems like the way to go, both from the user perspective and the thing you want to do.
Like: The columns are already present in SW2. They just extend past the equave and aren't editable. It doesn't feel like a stretch to imagine the equave-extended tuning table living on a separate tab, and the main page having all the same columns but within the equave, and each one editable.

And from your language design perspective, it might still make sense. What you want to build is a unified programming/logical language for pitch space as it relates to music, right? And many of the difficulties you've encountered are collisions between the previously established languages. Witness the 1,5 for example.
But is there any reason you have to allow those to be mixable in the same expression? Maybe I'm missing something, but it looks to me like there isn't. If your language contains a0 parser which translates back and forth between any of the data formats it supports. Then the users may always able to access whichever format is most convenient for their needs.
So like, say I have a Just interval and I want to add 50 cents to it, but I don't know how many cents it is. I type 19/14 in the ratio column. It fills the other columns up. I click in the cents column and after... (checks xen-calc) 528.69 I just type in "+50".

frostburn

The impression I got from tristan bay is that having multiple syntaxes introduces unnecessary confusion.

A.Liv

Bob's your uncle or whatever. 🙂
A.Liv

I think (tell me if I'm wrong Tristan) that that comment was in response to the "have a legacy switch for the old behavior" proposal.
Which is a bit different I guess.
IDK, you can also tell me if you feel like I'm heading somewhere you don't wanna go at all with this. But I'm getting excited about this idea now. 😆

frostburn

I will implement something and get back to you. This is getting quite theoretical and the program will inform the practice once I get a view of the Occam's Razor in code.
A.Liv

(I started typing and will finish before I respond to that, just making sure I get the point across).
Like, just imagine you have columns for: frequency in Hz, ratio, Cents, FJS. And you can type in any of them to modify the degree in question, including sensible interval arithmetic in each one. And then the scale generation tools along with that. Like xen-calc and SW2 in one, that'd be critically awesome IMO.
For sure, see where the path takes you! Again, just shedding ideas. The idea of separating each format out into columns seems to me like it may serve well your causes of designing a sensible language while also putting something out that the community will like.
But you never know until you've tried indeed.

frostburn

The main issue I see with that is that rational Hz / JI results in irrational cents or that rational cents cannot be represented in FJS.
There's no FJS symbol for 1\4 even with my extensions as that would already be a quarter-interordinal.

A.Liv

True.
Doesn't that problem need a solution anyway though? Like, if SonicWeave is supposed to let these data formats all coexist, it needs to know what to do when input and output are incompatible doesn't it?
Like, what FJS interval is 3 cents?

frostburn

In my minds eye I'm seeing all the columns displayed all the time. I guess it's fine if they go blank when the result is not representable.

A.Liv

Oh yeah haha.
Simple as that I guess. 🙂
And in the more general sense, like IDK, imagine me interacting with SonicWeave through a command line tool or something, if I ask it the FJS name of 3 cents it just throws an error.
Makes sense.

I guess you'd still need a way to distinguish the data formats in the language somehow. But that's ok! It's probably more important for a UI to be scala-compatible than for the whole language to be.
frostburn

Cents would be nice to see all the time, but I could come up with a void operator so that the application can show an approximation and indicate it as such. If the user edits out the voidness, it's on them that they lost representability in the other columns.

A.Liv

And yeah this makes sense to me. In the ODDsound UI I linked before, you type in ratios it gives you the cents next to it. You add one cent the ratio changes from 3/2 to 1.50whatever.
And won't change back. But that's usually alright.

0.5° 🌜

about the tuning table: I use it sometimes:
for looking at what notes exactly I do play;
for looking at intervals that span more than one period, like when a scale repeats in fifths but I want to look what’s with the octaves there
it can be probably all refactored somewhere else
I’d like a column which presents interval as a ratio (maybe plus un-prime offset displayed in cents) or, if prime powers are fractional, then probably as a monzo or something compact if there’s reasonable choice. SW2 doesn’t have that
then it covers functions of a ratio column but also retains some readability (I hope). Maybe it should be configurable. Maybe there’s a list of various types of columns which the user can toggle on and off in the preferences

A.Liv

“If ratio has few enough digits show n/m, else show the monzo, unless it’s like, a 1 and then nineteen 0s and another 1, then just show the factors”.
Is a good idea IMO.
And yeah the tuning table is useful for some things. But could go on a separate tab like the matrix maybe?

0.5° 🌜

maybe the matrix can be adjusted to show more steps; also maybe it could even highlight which steps and intervals between them are playing right now :galaxybrain~2:

Having an option to showing inverses of the usual ratios in a column could be useful as those correspond to ratios of wavelengths.

So to be clear: I don't think that slash notation ratios should ever be taken to refer to wavelength ratios. The proposal was that enumerations can be shown as both perspectives.
I see no benefit to doing this with slash notation, since slash notation by its nature describes dyads (typically the dyad of the note in question against the reference), and the frequency<->wavelength relationship is so trivial there.

I was referring to the "ratio" column in the tuning table. It's a decimal field so multiplicative inversion makes a significant difference.

Ah right, I see. IDK, I think it's fair to assume users looking in that column want frequency ratios 99.9% of the time, and the one time they want wavelengths (say for a fretboard design project or something) that would probably be better dealt with separately to minimize confusion potential.
Whereas showing harmonic/frequency and subharmonic/wavelength enumerations (in analysis tab or elsewhere) both, side-by-side, seems like it'd offer some interesting insight that people maybe wouldn't have considered. But in a way with less potential for confusion.


Add server-render support for CSS-in-JS

Fixes #538 #584
This is ready for API review. Once we're OK with the general direction, I'll put some time into polishing it up. It is not tied to any particular CSS-in-JS implementation. Rather, it provides some hooks that a caller can use to register a server stylesheet object to render into, and then fetch data out of that stylesheet.
You could also use this API for some generic data set up, such as #487. This is mostly intended as an escape hatch to run arbitrary Javascript on the same engine that is about to render the component.
This needs a bit more testing with other CSS-in-JS libraries (such as aphrodite). I only tested it with styled-components.
We could make this API a bit less generic, and register the functions for the lifetime of the app instead of passing them in to the Html.Render helper method. The downside is there needs to be a way to access the current ViewBag for the request so that the stylesheet can be written on the rendered layout.

I'd be interested in seeing this feature added. The first thing I did on my latest React.net project was to explore CSS-in-JS options and soon ran into issues (and then #538 's explanation).
I'll see if I can find some time to test with my preferred styling library.

I haven’t forgotten about this, just have spent my recent evenings and
weekends doing home remodel work :) I should have time this weekend
On Wed, Aug 29, 2018 at 14:35, Miles Rausch<EMAIL_ADDRESS>wrote:

I'd be interested in seeing this feature added. The first thing I did on
my latest React.net project was to explore CSS-in-JS options and soon ran
into issues (and then #538
https://github.com/reactjs/React.NET/issues/538 's explanation).
I'll see if I can find some time to test with my preferred styling library.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/reactjs/React.NET/pull/589#issuecomment-417113849,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA5hFr4vAbNqnmg4dtN0gEE2JdP1Xc2Wks5uVwkjgaJpZM4WMQiL
.

I restructured how the render helper functions got passed around and moved them into an interface and some helper classes. This PR demos using both styled components and react router in the same project with these new helpers, and allows new ones to be added either inside this library or by third parties easily.

This has been in production for about a week on one of my team's projects and has worked well. I'm going to push up some method documentation, add some tests, and then merge it. I'd like to get some docs pages added as well detailing how to use this feature but that might come after this PR is merged.

Opening a separate PR to update the samples

Would be nice with en example on how to do SSR with https://github.com/cssinjs/react-jss I'm having a hard time wrapping my head around this.


[15.0.x] Negative metrics raise exceptions in micrometer+prometheus Closes #13235

Backport: https://github.com/infinispan/infinispan/pull/13393
Closes #13235

@karesti is failing to compile :/


Overwriting is not work!

I added 1 file in resources/js/components like docs. But it's not work. Code in app.js is also not work.
Please write the docs more clearly, @Tarpsvo

Hello!
If you have a step by step way of reproducing the issue you're facing, please provide one, otherwise we can not reproduce the issue you are facing.
The package documentation is sufficient for our team of developers to understand how it works. If you need more instructions on how to use this - feel free to open up a pull request to enhance new developer experience for other users after thoroughly introducing yourself on how it works.
If you are enjoying working with our open-source packages, please also consider sponsoring our team so that we can keep maintaining our software!
I will however close this issue right now as there is no tangible issue mentioned in this thread.


How to mention user in message?

I want to have message like Welcome @username I've tried this var mention = "[" + firstName + "](tg://user?id=" + str(newUserId) + ")" but it returned error error: [polling_error] {}
var mention = "[" + firstName + "](tg://user?id=" + str(newUserId) + ")"
bot.sendMessage(chatId, "Welcome " + mention);

I could'nt tell you the reason for the error, since you do not specify how the complete error log is. What I can do is help providing you a code snippet for such events like welcome and farewells.
bot.on('message', function(msg){
    var chatId = msg.chat.id;
    var chatitle = msg.chat.title;
    if (msg.new_chat_members != undefined){
        var nameNewMember = msg.new_chat_member.first_name;
        bot.sendMessage(chatId, "Hi  " + nameNewMember + ", Welcome to te jungle" + chatitle);
    }
    else if (msg.left_chat_member != undefined){
        var nameLeftMember = msg.left_chat_member.first_name;
        bot.sendMessage(chatId, nameLeftMember + " Bye bye")
    }
});

The property that contains the username of the user is in msg.new_chat_member.username and msg.left_chat_member.username but remember, username isnt always available because some users dont set it in them acounts. (In case the new user does'nt have a username, it will return undefined)

@Seventty  thanks I've solved it by this code
var mention = `<a href="tg://user?id=${newUserId}">@${firstName}</a>`;


Hadoop index task occasionally fail

Hi all,
I'm having an issue where some of our Hadoop indexing tasks fail (most of our tasks are successful). I think it happens when the input / resulting segment size is large (maybe some memory limit is exceeded), but I can't find any hint to that in the exception. The task reaches map 100% reduce 100% and then fails.
This is the error from the task log:
2017-11-23T04:14:20,685 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job -  map 100% reduce 100%
2017-11-23T04:15:07,856 INFO [HttpPostEmitter-1-0] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://x.y.z.w:7000
...
2017-11-23T04:26:23,588 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job - Job job_1509360417142_3862 failed with state FAILED due to: Task failed task_1509360417142_3862_r_000000
Job failed as tasks failed. failedMaps:0 failedReduces:1

2017-11-23T04:26:23,659 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job - Counters: 42
	File System Counters
		FILE: Number of bytes read=8892350534
		FILE: Number of bytes written=17857749481
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=23059
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=148
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=0
		S3: Number of bytes read=3482384404
		S3: Number of bytes written=0
		S3: Number of read operations=0
		S3: Number of large read operations=0
		S3: Number of write operations=0
	Job Counters
		Failed reduce tasks=4
		Launched map tasks=74
		Launched reduce tasks=4
		Rack-local map tasks=74
		Total time spent by all maps in occupied slots (ms)=188764650
		Total time spent by all reduces in occupied slots (ms)=2024207616
		Total time spent by all map tasks (ms)=4194770
		Total time spent by all reduce tasks (ms)=10542748
		Total vcore-milliseconds taken by all map tasks=4194770
		Total vcore-milliseconds taken by all reduce tasks=10542748
		Total megabyte-milliseconds taken by all map tasks=6040468800
		Total megabyte-milliseconds taken by all reduce tasks=64774643712
	Map-Reduce Framework
		Map input records=102196318
		Map output records=102196318
		Map output bytes=48889085383
		Map output materialized bytes=8949241057
		Input split bytes=23059
		Combine input records=0
		Spilled Records=203673175
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=101204
		CPU time spent (ms)=5133010
		Physical memory (bytes) snapshot=46403289088
		Virtual memory (bytes) snapshot=190116319232
		Total committed heap usage (bytes)=39559626752
	File Input Format Counters
		Bytes Read=0
2017-11-23T04:26:23,662 INFO [task-runner-0-priority-0] io.druid.indexer.JobHelper - Deleting path[/tmp/druid-indexing/bandwidth-usage/2017-11-23T010002.727Z_45281c93490b4a659656249535b63608]
2017-11-23T04:26:23,687 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_bandwidth-usage_2017-11-23T01:00:02.713Z, type=index_hadoop, dataSource=bandwidth-usage}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:211) ~[druid-indexing-service-0.10.0.jar:0.10.0]
	at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:223) ~[druid-indexing-service-0.10.0.jar:0.10.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.0.jar:0.10.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.0.jar:0.10.0]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:208) ~[druid-indexing-service-0.10.0.jar:0.10.0]
	... 7 more
Caused by: io.druid.java.util.common.ISE: Job[class io.druid.indexer.IndexGeneratorJob] failed!
	at io.druid.indexer.JobHelper.runJobs(JobHelper.java:369) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]
	at io.druid.indexer.HadoopDruidIndexerJob.run(HadoopDruidIndexerJob.java:95) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]
	at io.druid.indexing.common.task.HadoopIndexTask$HadoopIndexGeneratorInnerProcessing.runTask(HadoopIndexTask.java:276) ~[druid-indexing-service-0.10.0.jar:0.10.0]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:208) ~[druid-indexing-service-0.10.0.jar:0.10.0]
	... 7 more
2017-11-23T04:26:23,693 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_hadoop_bandwidth-usage_2017-11-23T01:00:02.713Z] status changed to [FAILED].
2017-11-23T04:26:23,695 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
  "id" : "index_hadoop_bandwidth-usage_2017-11-23T01:00:02.713Z",
  "status" : "FAILED",
  "duration" : 12375913
}

Any help is appreciated!
Thanks,
Eran

you can try to adjust memory size of yarn in the yarn-site.xml,and maprduce memory
 "jobProperties" : {
        "mapreduce.reduce.java.opts" : "-Xmx10240m",
        "mapreduce.map.java.opts" : "-Xmx10240m"
      }

this code wile adjust the memory size of mapreduce,
sorry,i'm english isn't good......

Thanks @jdlzy! I currently have much lower values (512M / 2G) but do you know this is really the problem? because I don't see any indication of this in the exception...

I think,you can try to less data(eg:<20m or <10m),if  that's the case  success,that's your memory to lower。
but the (512M / 2G) is too lower

"jobProperties" : {
"mapreduce.reduce.java.opts" : "-Xmx10240m",
"mapreduce.map.java.opts" : "-Xmx1024m"
}
this configuration in the index file,you should amplify "mapreduce.reduce.java.opts",

Thanks! I will try it out

Just to update this thread that the problem seems to be solved, didn't have a failed task for a few days. However I made 3 changes in parallel, so not entirely sure yet which ones were actually required -

Increased the map/reduce Java heap limit (as suggested by @jdlzy), 10g didn't work for me, I'm now using 1g for map and 3g for reduce.
Reduced the number of concurrent Hadoop tasks that I'm submitting
Enabled hashed partitioning (targetPartitionSize)


Tagging of Volume

Customer would like to have capability to tag volumes for business units, workloads, projects, cost center etc etc.
In customer environment volume names are using artificial, random names during the provisioning process. For proper identification they would like tags for project, cost-center etc
Q: Can you please advise us on how to add custom tags to time series for volumes. Like BusinessUnit, Project, UsageModel etc…
A: Prometheus supports labels (which can be treated like tags). A counter can have many labels.
Typically we assign labels to counters like:
•	datacenter
•	cluster
•	vserver
•	volume
•	…
Labels get assigned during ingest into the database. You cannot (like with all TSDBs) add tags
or labels afterwards. If you want to add custom labels to volumes as outlined below, there
are actually three options to do that:

you use the comment filed in ONTAP itself – you need to create a plugin in Harvest
to decode the comment field and create labels for the volume. We would recommend
to create a new counter per volume which holds all these labels  - this is more efficient
and can be used for filtering in Grafana
You load this new volume counter manually into Prometheus using the file exporter
capability of Prometheus node exporter
You create a new collector which generates these counters

It’s more the question who and when do you want to assign these tags to the volume
For option 1: Can the netapp team create that plugin… I am sure this is something that all your customer base will find useful.
Q: Sorry if this is a repeated question, but can you provide 1-2 examples of the comment section and the labels you want to extract?
A: How about the following
Comment= “ Label1: Label2: Label 3:  Label4: ………”
Example Comment=” Business_group:XYZ  project:my_project  user1:myuser1 user2:my_user2 … and many more”
Each label:label data separated by spaces.. You can interpret as many labels or set a limit of 10 labels.
Label data and label names cannot have any special characters other than “_ or – “ and alphanumeric characters only
We can start with volume management. But if easy to implement for other objects we should. But volume is priority
Q: If the Zapi object does provide the comment as a metric, then it's easy and probably no need to write a new plugin for another object.
One last question that might sound a bit strange, but makes a big difference how the plugin would be implemented: how are you planning to use these labels? Do you want to be able to filter/select the metrics based on these labels? Or is it sufficient for you to get a table showing these labels (e.g. which volumes have which owners, projects, etc.).
A: We expect to filter based on these metrics and do various operation. Also we plan to implement alerting and monitoring also based on these filters.
Example: If utilization >90 for businessunit:xyz  then send email or raise alert  etc…

Reference of the volume "comment" field:
http://web.eng.netapp.com/engineering/rlse/DOT/devN/final/bedrock/export/common/zapiprod/web_ontap_cluster/volume/index.html#volume-attributes and search for "comment" (internal to netapp). It has a restapi mapping.

Comment= “ Label1: Label2: Label 3: Label4: ………”
Example Comment=” Business_group:XYZ project:my_project user1:myuser1 user2:my_user2 … and many more”
Each label:label data separated by spaces.. You can interpret as many labels or set a limit of 10 labels.

@damalik, sorry I lost track of our conversation about this issue, but this should not require any new feature. Splitting key-value pairs from a label is possible with the built-in plugin LabelAgent. All you have to do is edit the templates of the Zapi/ZapiPerf objects where you are inserting this comment label.
Assuming that is the case for the Volume object, you would need to change the volume.yaml template. There are two sections that you need to change.
1. - In the plugins section, add a rule for the LabelAgent plugin. The following should work for the example you brought above:
in the original template, the plugins section looks something like this:
plugins:
  - Volume

change that to:
plugins:
  Volume
  LabelAgent:
    split_pairs: comment ` ` `:`

2. - change the export_options section to make sure the extracted labels are included in the final metric. Again, I take here your example:
the section should originally look something like this:
export_options:
  instance_keys:
    - volume
    - node
    - svm
    - aggr
    - type

you should change it to look something like this:
export_options:
  instance_keys:
    - volume
    - node
    - svm
    - aggr
    - type
    - Business_group
    - project
    - user1
    - user2

if the set of labels that need to be extracted from comment are not pre-defined, you can simply change the export_options section to:
export_options:
  include_all_labels: true

We'll use this issue to update the documentation and validate this. Thanks @vgratian

if the set of labels that need to be extracted from comment are not pre-defined, you can simply change the export_options section to:
export_options:
  include_all_labels: true

Here is feedback from customer, something does not seems to be working
Hi Deepak
I have update volume.yaml file as suggested below.
Updated the comment for a volume as below.
xyz-clu01::> vol modify macharya_test -vserver xyz-clu01-scr04 -comment "business_unit:IT project:unix workflows:test owner:macharya bu_project:IT"
Volume modify successful on volume macharya_test of Vserver xyz-clu01-scr04.
xyz-clu01::> volume show macharya_test -vserver xyz-clu01-scr04 -fields comment
vserver         volume        comment

xyz-clu01-scr04 macharya_test business_unit:IT project:unix workflows:test owner:macharya bu_project:IT
Restarted the poller after this and searched for this volume in Prometheus.
Still don’t see comments added displayed as labels. How long we need to wait for the labels to show up? Is there any typo in my volume.yaml file?

Hi @demalik, I see that there were a few mistakes in my suggestion, below are the steps once again, this time they are correct.
Main mistake was to suggest to use the ZapiPerf collector (it does not provide the comment section as a counter), so instead please change the volume.yaml template of the Zapi collector:
(1) First make sure that "comment" is collected, add it under the counters section, under the volume-id-attributes:
counters:
  volume-attributes:
    - volume-autosize-attributes:
      - maximum-size
      - grow-threshold-percent

    - volume-id-attributes:
      - ^^instance-uuid             => instance_uuid
      - ^name                       => volume
      - ^node                       => node
      - ^owning-vserver-name        => svm
      - ^containing-aggregate-name  => aggr
      - ^style-extended             => style
      - ^comment                    => comment  # only this line needs to be added
     ....

(2) Under the plugins section add the exact same split_pairs rule, the section will look like this:
plugins:
  LabelAgent:
    value_mapping: status state online `1`
    exclude_equals: style `flexgroup_constituent`
    replace: style style `flexgroup_constituent` `flexgroup`
    split_pairs: comment ` ` `:`  # only this line is added
  Aggregator:
    volume<style=flexgroup>volume node,svm,aggr,style

(3) Finally under export_options, add the labels that you expected to be extracted, e.g.:
export_options:
  instance_keys:
    - volume
    - node
    - svm
    - aggr
    - style
    - business_unit
    - project
    - owner
    - bu_project
  instance_labels:
    - state

Here is how I tested it:

Added comment:

# Using the ONTAP CLI:
shopfloor::> vol modify -volume vgratian_tmp_will_delete -vserver svm0 -comment "business_unit:IT project:unix workflows:test owner:macharya bu_project:IT"
Volume modify successful on volume vgratian_tmp_will_delete of Vserver svm0.

Started Harvest:

harvest start shopfloor -f --config harvest_.yml -c Zapi -o Volume

Tested the metrics:

sanjunipero>$ curl localhost:5551/metrics | grep vgratian_
volume_labels{datacenter="WDRF",cluster="shopfloor",volume="vgratian_tmp_will_delete",node="shopfloor-02",svm="svm0",aggr="aggr02",style="flexvol",business_unit="IT",project="unix",owner="macharya",bu_project="IT",state="online"} 1.0
volume_size{datacenter="WDRF",cluster="shopfloor",volume="vgratian_tmp_will_delete",node="shopfloor-02",svm="svm0",aggr="aggr02",style="flexvol",business_unit="IT",project="unix",owner="macharya",bu_project="IT"}<PHONE_NUMBER>
volume_size_total{datacenter="WDRF",cluster="shopfloor",volume="vgratian_tmp_will_delete",node="shopfloor-02",svm="svm0",aggr="aggr02",style="flexvol",business_unit="IT",project="unix",owner="macharya",bu_project="IT"}<PHONE_NUMBER>
....

I will take a look for the case when the extracted label set are not predefined, not sure if we can do that with the currently available plugins..

include_all_labels: true should work, but it will add a lot of redundant labels, including the original comment label.

@Hardikl, sorry, I hope you didn't already spend time on this, felt obliged to correct my mistakes

@vgratian I haven't started looking into this yet.

Closed as part of #309. Feel free to reopen @madhusudhanarya if there's more to cover here. I added this information to the wiki.


Generate TLS certificates as part of the test

Motivation
Follow up to #5347: instead of re-using generated templates create them as part of the test.
Changes

Update test to generate the certificates it needs

Test Plan

existing tests pass

TODO

[x] Explain motivation or link existing issue(s)
[x] Test changes and document test plan
[x] Update documentation as needed
[x] Update changelog as needed

README changes appear unrelated

They have been taken from the deleted README.md in the testdata folder since I didn't want this information to get lost and that information will be relevant for smeshers running a remote PoST service, so I put it into the main README.md 🙂

bors merge


css property -ms-flex-direction doesn't get into assembly

Describe the bug
css property -ms-flex-direction doesn't get into assembly after npm run build. css property -ms-flex-align and -ms-flex-pack get into the assembly. I tried to find the mention of -ms-flex-align and -ms-flex-direction in the dependencies and found only in csstree, but the edits had no effect on the result.
Did you try recovering your dependencies?

I put together a test project with a minimum of code in order to reproduce the error. I used the latest version of the node and npm
npm --version
6.13.7
Which terms did you search for in User Guide?

My searches in the user manual and the Internet did not help. I tried looking for -ms-flex-direction and react create app. I tried looking for -ms-flex- in the dependency code and found a mention in the csstree package, but my attempts to fix the code did not help.
Environment

I tried to reproduce the problem on windows and mac. On the latest and stable version of the node. The result is always the same.
npx create-react-app --info
npx: установлен 99 в 6.693s
Environment Info:
current version of create-react-app: 3.4.0
running from /Users/temoffey/.npm/_npx/53798/lib/node_modules/create-react-app
System:
OS: macOS 10.15.2
CPU: (4) x64 Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
Binaries:
Node: 12.14.1 - /usr/local/bin/node
Yarn: Not Found
npm: 6.13.7 - /usr/local/bin/npm
Browsers:
Chrome: 79.0.3945.130
Firefox: 72.0.1
Safari: 13.0.4
npmPackages:
react: ^16.12.0 => 16.12.0
react-dom: ^16.12.0 => 16.12.0
react-scripts: 3.3.0 => 3.3.0
npmGlobalPackages:
create-react-app: Not Found
Steps to reproduce

Run 'npm init react-app my-app';
Edit file 'src/index.css' add string 'div.test {
-ms-flex-align: end;
-ms-flex-direction: column;
}';
Run 'npm run build';

Expected behavior

File '/build/static/css/main.********.chunk.css' contains a string 'div.test{-ms-flex-align:end;-ms-flex-direction:column}'
Actual behavior

File '/build/static/css/main.********.chunk.css' contains a string 'div.test{-ms-flex-align:end}'
Reproducible demo

https://github.com/temoffey/react-issue

The problem is still relevant

The problem is still relevant

The problem is still relevant


assumeRoleWithWebIdentity with webIdentityTokenCommand

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details
We love the new assumeRoleWithWebIdentity feature where we can use an OIDC token to authenticate to AWS. Currently it supports either webIdentityToken or webIdentityTokenFile in the config with the value of the token or a filename of the file that contains the token. It would be great if we can get a third option webIdentityTokenCommand (or something similar) that executes a shell command to get the token.
We would use that to have pulumi invoke gcloud auth print-identity-token to get a GCP identity token as we have setup AWS to trust Google cloud tokens.
This feature could also be used if people have other complex methods to get a (fresh) id-token.
An alternative would be if we can get the token from our pulumi code itself (typescript in our case) and somehow inject it into the aws provider.
Affected area/feature
This affects the "Authenticate with WebIdentity and OpenID Connect (OIDC)" feature of @pulumi/aws

Thank you for the suggestion and the feedback @wvanderdeijl
I'm curious which language are you using to drive Pulumi?
I am thinking perhaps of suggesting to try explicit providers here: https://www.pulumi.com/docs/concepts/resources/providers/#explicit-provider-configuration
In a language like TypeScript it may be possible to reach out to a shell command before passing the result as an input to the AWS resource. If you are using YAML it may be little trickier, but have you seen the command provider?
https://www.pulumi.com/registry/packages/command/api-docs/local/command/
With this little bit of indirection it might be possible to shell out to get the token and then pass it to the explicit provider to accomplish your goal.

Thanks for the tip. I tried something from typescript. Not even with a shell command but just getting the token from typescript:
import * as aws from '@pulumi/aws';
import { all, output, secret } from '@pulumi/pulumi';
import { auth } from 'google-auth-library';

const serviceAccount = output('my-service-account@my-project.iam.gserviceaccount.com');
const audience = output('*****');
const awsRoleArn = output('arn:aws:iam::999999999999:role/pulumi-aws-federation');

const token = secret(
    all([serviceAccount, audience]).apply(async ([serviceAccount, audience]) => {
        const client = await auth.getClient();
        const response = await client.request<{ token: string }>({
            method: 'POST',
            url: `https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${encodeURIComponent(
                serviceAccount,
            )}:generateIdToken`,
            data: {
                audience,
                includeEmail: true,
            },
        });
        return response.data.token;
    }),
);
const provider = new aws.Provider('aws', {
    region: 'eu-west-1',
    assumeRoleWithWebIdentity: {
        roleArn: awsRoleArn,
        webIdentityToken: token,
    },
});
new aws.s3.Bucket('my-bucket-53efcbd4499a768d', { tags: { foo: 'bar' } }, { provider });

This works, although every pulumi preview shows the aws provider as outdated due to the changing webIdentityToken. But the dependent resources (such as the S3 bucket) do not show up as outdated and are not touched during pulumi up.

I waited for a couple of days so the token in the provider in the pulumi state has expired. As expected removing both the provider and the s3 bucket from the pulumi typescript file will result in an error as the stale information from the state file is used:

  aws:s3:Bucket (my-bucket-53efcbd4499a768d):
    error: 1 error occurred:
    	* Cannot assume IAM Role with web identity: IAM Role (arn:aws:iam::611897761625:role/pulumi-aws-federation) cannot be assumed with web identity token.

    There are a number of possible causes of this - the most common are:
      * The web identity token used in order to assume the role is invalid
      * The web identity token does not have appropriate permission to assume the role
      * The role ARN is not valid

    Error: failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, https response error StatusCode: 400, RequestID: 81525888-7396-431b-ab19-440fb6c93ae7, ExpiredTokenException: Token expired: current date/time<PHONE_NUMBER> must be before the expiration date/time<PHONE_NUMBER>

Removing only the s3 bucket resource from the pulumi stack, but keeping the provider does work. Pulumi first updates the provider with a fresh token and than uses that updated provider to remove the s3 bucket.
With the caveat that you should not remove the provider in one go, we can use this workaround. However, I still feel having a webIdentityTokenCommand setting would be a preferable long term solution.


fix: support waiting for jobs with ttlSecondsAfterFinished

Rely on job events instead of polling while waiting for job readiness. This way job completion can be observed even if the job was removed because of expired TTL
Signed-off-by: Vladislav Koriakov<EMAIL_ADDRESS>
What this PR does / why we need it:
This PR is intended to fix the behavior of '--wait-for-jobs' flag when jobs with ttlSecondsAfterFinished present (using lower values in particular). Previously used polling to check for job readiness might wail as the job might have reached its TTL and was already removed by the time polling happens. Instead of polling there is event watch for jobs starting from particular resource version (after helm upgrade/install is performed) that can capture job completion.
Special notes for your reviewer:
The fix was tested using minikube and helm chart with jobs (with and without ttlSecondsAfterFinished set) and different scenarios, e.g. two successful jobs, one failed and one successful job, etc.
If applicable:

[x] this PR contains documentation
[x] this PR contains unit tests
[ ] this PR has been tested for backwards compatibility

Link to ttlSecondsAfterFinished  docs

Thanks for the PR 👍 . I think generally this is a nice idea. Watching/being event driven is better than polling.
One nicpick, is that unless I am mistaken, watch requires that additional rbac verb. Strictly thus, this change isn't backwards compatible. I also don't know offhand if helm expects the operator to have watch privs (which would make this point moot)
For the implementation, two thoughts:

The way the code is structured, I think means that the watch isn't invoked until waiting for other resources to complete. The job could have already been deleted by the ttl by this time. So while the code is an improvement, I don't think it quite covers all the scenarios.
An (integration) test for coverage would go a long way. The manual testing is appreciated, but without (automated) test coverage, ensuring this functionality/feature works into the future becomes difficult.

Hi,
regarding backward compatibility: is there any suggestion on making this change backward compatible? The only approach that comes to my mind is to check if supplied job spec has 'ttlSecondsAfterFinished' field set, if it's not - use existing implementation with polling.
Points on implementation:

Shouldn't be a problem, as watch is registered with resourceVersion obtained after job creation, all subsequent events will be observed. See detection of changes doc.
Is there any existing examples of integration tests in the project I can refer to? Or any suggestion on how such test should look like?

Thanks.

regarding backward compatibility: is there any suggestion on making this change backward compatible?

I think worth joining the weekly developer call to discuss. I raised this as a "technicality", I'm not sure it is a blocker. But for example, a new cli flag to opt-in to watching would also work.

Shouldn't be a problem, as watch is registered with resourceVersion obtained after job creation, all subsequent events will be observed. See detection of changes doc.

ah, nice. thanks for pointing that out.

Is there any existing examples of integration tests in the project I can refer to? Or any suggestion on how such test should look like?

I believe there are (I'm new to the helm project). Also worth asking on the call I think.

regarding backward compatibility: is there any suggestion on making this change backward compatible?

I bought this PR up at the helm weekly developer meeting. There was interest, but the additional RBAC requirement was a concern. One suggestion was to check the users auth (e.g. same as kubectl auth can-i watch jobs) and only watch if that check is successful. And fall back to the existing poll (and perhaps print a warning) otherwise.
(still need to dig up test examples)

Thank you.
Will proceed with the PR when more free time is available.
Let me know if there are integration tests examples I can refer to.

Added unit test, slightly refactored waitJobsReady.
Regarding RBAC concern: it shouldn't be an issue from my POV, there already exists event watching functionality for hooks that apparently requires the same permissions for watching events, see client.go in kube package. I couldn't find any explicit check for permissions before that code is executed, so this PR shouldn't introduce any (new) RBAC issue.
Let me know if explicit permission check is still desired before 'waitForJobs'.
If so, I think it also makes sense to create a ticket to add one for hooks functionality.

Thanks for the feedback.

Overall, I (still) think it would be much better if Helm could switch to watching, rather than polling.
I agree, watching seems to be more reliable than polling for most (all?) of helm operations.

The main contention I think with the code changes here, is that there is a lot of boilerplate to support watching for batch/v1.Jobs, And perhaps, it would take significant refactoring to support other types.
I would argue the code here is written is similar fashion as the existing code.
'IsReady' of 'ReadyChecker' switches on different resource types, polls the status and calls appropriate resource method ('isPodReady', 'jobReady', 'serviceReady', 'deploymentReady' etc).
'getResourceWatch' introduced in this PR follows similar structure - switch statement on particular type of resource, implemented only for jobs for now, but later can be extended to support other types.

Between these two things, I feel like there is a much cleaner implementation for implementing wait-with-watch behavior. Which would scale to other types/scenarios.
Maybe, but outside of scope of the issue this PR is trying to solve.

It could be argued that Helm's wait for Jobs with ttlSecondsAfterFinished is already broken today. And so this fix is needed before a more comprehensive solution. But often the "quick fixes" become permanent, show limitations over time, but become ossified into the code's behavior. And hard to change/remove. So tend to not be desirable.
My goals were:

write little code (so less is affected by the change)
write code that fits the current structure (so less is affected by the change)
make ttlSecondsAfterFinished work

As a helm user I want (need) this feature to work, but if in order to do so there should be an architectural change - supporting watching for all resource types instead of polling - this sounds out of scope of this issue.
I, potentially, would be interested in contributing general poll->watch replacement, but definitely not as part of fixing 'ttlSecondsAfterFinished'.


During a playbook run, if User go to some those options(:collections/:config/:images/) and stay until the playbook finished, an error will occur

During a playbook run, if User go to some those options(:collections/:config/:images/) and stay until the playbook finished, an error will occur

run a playbook using the TUI
while the playbook is running, go to one of those options: :collections/:config/:images/
stay until the playbook finished
an error occurs.

Traceback (most recent call last):
  File "/home/yagosilva/venvs/navigator/bin/ansible-navigator", line 33, in <module>
    sys.exit(load_entry_point('ansible-navigator', 'console_scripts', 'ansible-navigator')())
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/cli.py", line 137, in main
    return_code = run(args)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/cli.py", line 81, in run
    wrapper(ActionRunner(args=args).run)
  File "/home/yagosilva/.pyenv/versions/3.9.0/lib/python3.9/curses/__init__.py", line 94, in wrapper
    return func(stdscr, *args, **kwds)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/action_runner.py", line 65, in run
    self._run_app(interaction)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/action_runner.py", line 74, in _run_app
    interaction = run_action(self.steps.current.name, self.app, self.steps.current)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/_actions.py", line 97, in run_interactive
    return action_cls(app.args).run(app=app, interaction=interaction)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/run.py", line 300, in run
    self._take_step()
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/run.py", line 516, in _take_step
    result = run_action(self.steps.current.name, self.app, self.steps.current)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/_actions.py", line 97, in run_interactive
    return action_cls(app.args).run(app=app, interaction=interaction)
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/config.py", line 130, in run
    self._calling_app.update()
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/run.py", line 772, in update
    self.write_artifact()
  File "/home/yagosilva/projects/ansible-navigator/ansible_navigator/actions/run.py", line 817, in write_artifact
    playbook_dir=os.path.dirname(self._args.playbook),
  File "/home/yagosilva/.pyenv/versions/3.9.0/lib/python3.9/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not Constants

This is because args is passed by reference to another action and modified there.  We keep the "previous entries" and restore them when exiting the action, will need to talk about how best to solve this, maybe self._args should also be a deepcopy?


Sitong Lu -- Add HW6 assignment file

HW6 assignment

@Tammylu the assignment expected location is wrong. Overall assignment looks good 9/10


Call Out a limitation : DataSourceInfo is not supported with SharePoint

Can someone, please update the article to call out a limitation? DataSourceInfo() does not seem to work with SharePoint. It has never been working for the last 3 years.
This is what @ebaron from Microsoft said in 2019:
v-xida-msft Community Support v-xida-msft Community Support

Currently, the DataSourceInfo() function is not supported in SharePoint List. The Product Team is investigating this issue, and plan to add support for this function.

Feedback from other users
@WonderLaura

If you watched my SharePoint Power Hour today, you may have noticed that I was surprised when the DataSourceInfo function did not give me accurate permission information about the currently logged in user’s access to a certain list in SharePoint.

Michael Glavina

Hi Audrie. I am trying to use DataSourceInfo for an Admin screen. The SharePoint Online site (which was created automatically by MS Teams) has a default owners permission group that can edit a DataSource list called Admin (not inherited - custom permissions). The site members can only Read the list (not inherited - custom permissions). OnStart --> Set(IsAdmin, DataSourceInfo(Admin,DataSourceInfo.EditPermission)) always returns true for all users. I have seen comments online about this going back to 2016. Is this a known issue? Am I doing something wrong?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d3184818-d8e7-0d02-924f-55f8a7a34110
Version Independent ID: 61267e2c-bfb1-9bda-d728-62329f3f535b
Content: DataSourceInfo function in Power Apps - Power Apps
Content Source: powerapps-docs/maker/canvas-apps/functions/function-datasourceinfo.md
Service: powerapps
Sub-service: canvas-maker
GitHub Login: @gregli-msft
Microsoft Alias: gregli

Thanks for contacting us. We have updated the docs and pushed it live. Closing the issue. If you need further help, please reopen or create a new issue.


Fix keygen script to copy truststore file to device

Fix keygen script to copy truststore file to creds/device.
Signed-off-by: Davis Benny<EMAIL_ADDRESS>

@DukeDavis12 Any changes required in the device source code to read this truststore?


Named colors are replaced to hex codes with incorrect offset ranges

Describe the bug
Named colors is replaced to hex codes with incorrect offset ranges.
To Reproduce
Steps to reproduce the behavior:

Click a color rec for a named color. (e.g. NAMED_COLOR_TABLE.put("pink", "#ffc0cb");)
Choose a color via a color chooser.
The characters of before and after the named color are also removed. (e.g. NAMED_COLOR_TABLE.put(#c2ffc0, "#ffc0cb");)

Expected behavior
Only the named color is replaced.
e.g. (NAMED_COLOR_TABLE.put("#c2ffc0", "#ffc0cb");)
Desktop (please complete the following information):

OS: Ubuntu 18.04
NetBeans 11.0
Version of the plugin: 0.12.1

Fixed.


add status code 400 for update endpoints in swagger.yml

Signed-off-by: allencloud<EMAIL_ADDRESS>I think in docker/master we should add status code 400 for endpoint node update and secret update.
First pic is for node update.

I test docker 1.12.x,
Second pic is for secret update:

- What I did

add status code 400 for endpoint POST /node/{id}/update;
add status code 400 for endpoint POST /secret/{id}/update;

- How I did it
- How to verify it
- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

ping @vdemeester @thaJeztah

oh, ugh, I was about to comment but was in the airplane already; discussing with @justincormack, but this 400 should probably be a 404 given that the requested resource doesn't exist?


Readonly tests added.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+2.5%) to 94.416%

Totals

Change from base Build<PHONE_NUMBER>:
2.5%

Covered Lines:
186

Relevant Lines:
197

💛  - Coveralls


Fix issues with exe/floe and various combinations of workflow and input

This commit cleans up the exe/floe interface for bare workflow/input pairs when passed in combination with --workflow and --input. This commit also makes the --workflow and --input params no longer legacy but supported.
The following are now all accepted ways to pass these parameters:
# Single workflow
exe/floe workflow1.asl
exe/floe workflow1.asl '{"input":1}'
exe/floe workflow1.asl --input '{"input":1}'
exe/floe --workflow workflow1.asl --input '{"input":1}'

# Multiple workflows
exe/floe workflow1.asl '{"input":1}' workflow2.asl '{"input":2}'
exe/floe workflow1.asl workflow2.asl --input '{"input":1}' # input is dup'd for each workflow

The following will raise errors:
exe/floe workflow1.asl workflow2.asl               # ambiguous as to workflow/input pair or 2 workflows, and so requires input
exe/floe workflow1.asl --workflow workflow2.asl    # invalid mixing of bare workflows and parameter
exe/floe workflow1.asl workflow2.asl '{"input":1}' # mismatched workflow/input pairs
exe/floe --input '{"input":1}'                     # missing workflow
exe/floe                                           # no parameters

also, can we drop --input and --workflow.
I left them in when introducing ARGV, but think it is time to drop them.
Thoughts?

No this PR makes them a) permanent (I use them personally and prefer them) and b) input, specifically, is useful for multiple workflows that take the same input cause you only need to specify it once

I could see maybe removing --workflow in a future PR, but --input has a use case.


ACM-2747 Update to release-2.8/backplane-2.3

Signed-off-by: Kevin Cormier<EMAIL_ADDRESS>

/unhold


Build 164 - Automatically add path for all HTTP/HTTPS services

Add the root path to the db paths table.

Closes issue #75


Revert "Add GitHub action to create PRs merging google -> main"

Reverts google/iree#2482
This was broken in several ways. I'll try fixing it up and resubmitting.

Whelp that restore button is super not working. Thanks GitHub

Going to use admin powers to just revert this. It's a thing I wrote that is broken and not depended on and currently creating errant PRs


Add Github Sponsor or other way to "tip"

I would love to support development financially.

Hi Mike!
Glad to hear you'd like to support development. However we're not trying to make money on the project and your activity already quite useful, thank you! If you'd like to do something else for the project, please just share it with your colleagues, we will be very grateful!

Will do! I will be following development for sure and thank the team for the excellent work. I wish I was stronger in Swift/SwiftUI to jump into some issues that prevent some more advanced used cases, like nested Grid preference issue #16.


New pkgconfig integration

Describe the Feature
The ParseFlags family of functions relies on picking specific flags out of the command line. This is non-portable and hard to maintain. A better approach would be to build a tool that knows how to make better use of the pkg-config command line API to pick out the functional groups of flags and connect them to the analogous SCons flags.
Required information

Per discussion on Discord
Version of SCons: N/A
Version of Python: N/A

Additional context
The idea was to write a new tool that understood how to use pkg-confg args like --libs-only-L to know what arguments should be attached to SCons LIBPATH variable, or --cflags-only-I to tie it to SCons CPPPATH. Something like:
pkgFlagsEnv = myenv.Clone()
pkg_config = pkgFlagsEnv.Tool("pkg-config")
pkgFlagsEnv.AppendUnique(
    PKG_CONFIG="/usr/bin/pkg-config",
    PKG_CONFIG_PATH=["/some/custom/place/to/find/packages", "/another/custom/place"],
    PKG_CONFIG_PACKAGES=[
        "foolib",
        ("barlib", "1.23+"),
    ],
    PKG_CONFIG_ARGS="--static",
    CCFLAGS=pkg_config.with("--cflags-only-other"),
    CPPPATH=pkg_config.with("--cflags-only-I"),
    LIBPATH=pkg_config.with("--libs-only-L"),
    LINKFLAGS=pkg_config.with("--libs-only-other"),
    LIBS=pkg_config.with("--libs-only-l"),
)

It is probably possible to provide a tools scoped dictionary to kwargs in the usual override suspects:
pkgFlagsEnv.AppendUnique(
    PKG_CONFIG...,
    **pkg_config.standard_overrides)

If it's a Tool then it would make more sense to interact with it via methods instead.
Something like:
if env.CheckPkg("somepackage", min_version="1.0.0"):
    #flags are added at this point
else:
    #this package isn't found and script is expected to activate fallbacks at this point.

of course there should be lower level methods too, those that return dict with CPPPATH etc instead of adding them on their own.

I wouldn't consider my thing above to be in any way proposing a specific API. Just outlining an idea, and I'm sure there are many other ways to achieve what I'm suggesting. I guess if I had to summarize that idea in words, it would be to create a SCons tool that knows how to invoke the configuration tool repeatedly in the different modes to extract the relevant flags and apply them to the associated SCons Variables, without any special knowledge of or interest in the flags themselves. I think this is an important thing to do because I think it is basically impossible to make the existing ParseFlags feature robust. The approach it takes will always make it both too specific (hardcoded knowledge of certain flags) and incomplete (lack of awareness of other flags).

There already exists this api: https://pypi.org/project/pkgconfig/ . Can scons adopt it as a dependency?


Keycloak Export

Fixes #919
Changes:

Added a scripts/keycloak-export.py script to export users/groups from Keycloak
Documentation added to 'Manual Backups'
Also talk about JupyterHub DB backup in the docs

Types of changes
What types of changes does your code introduce?
Put an x in the boxes that apply

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds a feature)
[ ] Breaking change (fix or feature that would cause existing features to not work as expected)
[x] Documentation Update
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[ ] Other (please describe):

Testing
Requires testing

[ ] Yes
[x] No

In case you checked yes, did you write tests?

[ ] Yes
[ ] No

Further comments (optional)
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered and more.

BTW the 'dead link' detected by Vale won't be dead when this PR is merged!


JSMin is unsupported, consider replacing with JShrink

From the JSMin page:

This project is unmaintained. I stopped using it years ago. You shouldn't use it. You shouldn't use any version of JSMin. There are much better tools available now.

Consider replacing it with JShrink :-)

:+1:


Hardware layer optimization has drawing problem with webview

the "Explicit use of hardware layers for smoother scrolling" patches breaks rendering for webview.
Example:
Have two fragments:

RIGHT: main content with webview.
LEFT: menu content.

LEFT is hidden by default.
A) Load a webpage in webview in RIGHT and perform sliding menu to show LEFT.
B) So far so good. LEFT is showing and a small portion of RIGHT is showing at the right edge.
C) Now swipe left to hide menu.
D) Problems happens here. LEFT menu slides and hides to left which is good. But RIGHT slides into view with the "hidden" portion showing white blank. After slide completes, the "hidden" blank part of webview redraws to fix itself.
There appears to be a problem with webview drawing and hardware layers with sliding menu.

Same issue as @diegomontoya .


Cannot cull the village anvil ding sound.

Mod Version:
DynamicSurroundings-1.16.4-<IP_ADDRESS>
Forge Version:
36.1.10
Link to client log:
latest.log
Description:
I want to mute the sound environs:village.anvil because it's annoying and confuses my players, and I had muted it by adding "environs:village.anvil cull" to the Individual Sound Config config, in sndcntrl.toml. After restarting, I still hear the sound. After changing it to "environs:village.anvil cull 100" and restarting, I still hear the sound.
I thought it might be an incompatibility with Extreme Sound Muffler, but it kept making the sound after removing it. Here is my mod list: https://pastebin.com/bpFwfHmZ

Mod Version:
DynamicSurroundings-1.16.4-<IP_ADDRESS>.jar
Forge Version:
36.2.0
Description:
I'm experiencing a similar issue, except with environs:crickets and environs:deep_crickets (not sure it's limited to just these sounds to be honest).
If I add "environs:crickets block 0", "environs:deep_crickets block 0" to config/dsurround/sndctrl-client.toml, these sounds still play unless I open the "Individual Sound Configuration Settings" GUI in-game and toggle BLOCK on those sounds in there. Only works until the next client restart.


siri integration degraded, then completely stopped working with ios 14.6

I'm not sure if this an issue with the plugin or some other part of my setup.
My use case is a little strange. I have a zone on my open sprinkler controller hooked up to a relay that "pushes" my garage door opener button. I've had a rube goldberg shortcut that ended up calling a URL, but it's really unreliable with phone not always connecting to wifi quickly, etc.
I decided to try homebridge and this plugin, with my homepod as the home hub, and it started out working great with siri. I defined a "valve" called "Garage Door", could say "Hey Siri, turn on the garage door" as I pulled into the alley, and it worked every time.
At some point that stopped working, but I could say "Hey Siri, water the garage door" instead
Finally, controlling via Siri stopped working all together, I think when I upgraded to ios 14.6,
It still works if I manually use the button in the Home App.
Also, it seems like Siri knows about it a little because I can say "Hey Siri, What valves does my sprinkler system have", and Siri will answer "Garage Door is not being watered"
However, any attempt to control ends up with "Sorry, I didn't hear back from your devices. Make sure you're on the same Wi-Fi or have remote access setup in the Home App". This is the same on my iphone, ipad, and mac, but they all can then control using the interface in the Home app.
I don't know if there are any relevant logs.

Plugin Config:
{
    "bridge": {
        "name": "Homebridge F0B0",
        "username": "0E:89:E4:XX:XX:XX",
        "port": XXXXX,
        "pin": "934-XX-XXX",
        "advertiser": "ciao",
        "bind": [
            "ens3"
        ]
    },
    "accessories": [
        {
            "name": "Test",
            "stateful": true,
            "reverse": false,
            "time": 1000,
            "resettable": false,
            "accessory": "DummySwitch"
        },
        {
            "name": "Sink",
            "stateful": true,
            "reverse": false,
            "time": 1000,
            "resettable": false,
            "accessory": "DummySwitch"
        }
    ],
    "platforms": [
        {
            "name": "Config",
            "port": 8581,
            "auth": "form",
            "theme": "auto",
            "tempUnits": "f",
            "lang": "auto",
            "platform": "config"
        },
        {
            "password": {
                "md5": "XXXXXXXXXX"
            },
            "valves": [
                {
                    "name": "Garage Door",
                    "defaultDuration": 1
                }
            ],
            "host": "mysprinkler.wherever.com:8080",
            "pollInterval": 15,
            "platform": "HomebridgeOpenSprinklerApi"
        }
    ]
}

Environment:

Plugin Version: 1.1.0
Homebridge Version: 1.3.4
Node.js Version: v14.17.0
NPM Version: 6.14.13
Operating System: Ubuntu 20.04.2

First off, thanks for trying out my plugin! I've tried hard to make things as simple as possible.
Yeah your situation is rather tricky. The fact that you say that everything within the normal Home app works fine, leads me to believe that this is just an issue with Siri recognizing the names of things. A garage door is a specific type of HomeKit accessory, so it's possible that Siri is trying to look for one and isn't finding it. I do have a suggestion though that should work.
As of right now, the Home app doesn't allow you to set up scenes with an irrigation system. But, other apps do let you. I'd suggest you download the Eve app (It's free) and it'll allow you to create a HomeKit Scene for your "Garage door" valve. So you could name it something like, "Open Garage" and another one called "Close Garage". And then you can just use scenes to control that particular valve. In order to avoid any complications with Siri, I'd suggest renaming the garage valve to something unrelated to a garage.
Let me know if this works or doesn't work for you. But yeah when developing a plugin, we don't have any control on how Siri interacts with it. But I think setting up a scene will work well as it's worked well for me with other Homebridge accessories.

It's been 7 days and I haven't heard anything so I'm going to close this. Feel free to reach out if you have any other questions/concerns.

fwiw I'll reply since I stumbled on this... there's some kind of issue with Siri and sprinklers in current iOS. I can't get certain commands to work with RainMachine and have seen on Yardian forums that they've submitted bug reports to Apple re: this issue.
https://support.yardian.com/hc/en-us/community/posts/360078194752-Issues-with-Siri-Commands


Enhance regrid_data_plane to process Smoke and Dust from GOES-16 ADP datasets.

This is a request from the NOAA/EMC Aerosols group.  The regrid_data_plane tool was enhanced to process GOES-16 AOD data.  During a meeting at NCWCP on 9/11/2019, Ho-Chun Huang requested that regrid_data_plane be enhanced to also process data from GOES-16 ADP files.
See a description of the GOES-16 ADP data format attached.
GOES-16_ADP_readme_idl_code_20180328.docx
Also see and excerpt from the ADP user's guide attached.
GOES-16 ADP Users Guide extract.pdf
The ADP files are very similar to the AOD files, but slightly different.   The relevant data variables are named Aerosol, Dust, and Smoke.  Each of these fields contains 0's or 1's as a mask to define whether Aerosol, Dust, or Smoke is present at that pixel.  In addition, the DQF variable contains quality control flags.  However, the QC info is stored in the 8 bits of the DQF values as described in the document above.
When processing Smoke or Dust, parse the DCF bits to construct a QC value of 0, 1, or 2, for high, medium, and low quality, respectively.  Use the user-specified -qc flag command line option to determine which pixels should be used.  When doing the regridding step, determine how many of the pixels inside each grid box meet the -qc flag setting.  And then report the output value for that grid box as the average value of the good data pixel values.  The result should be a number between 0 and 1 indicating the percentage of the pixels in that grid box containing smoke or dust.
Ultimately, Ho-Chun plans to process both the ADP files to compute fields for smoke percentage and dust percentage.  Then he'll use those percentages to define masks when computing stats in Grid-Stat for the GOES-16 AOD values.  For example, he can compute the RMSE for AOD for grid boxes that are at least 50% smoke.
Even though Ho-Chun has no planned usage for the Aerosol field, we may as well include that as an optional output.
Here's a description of how the -valid_thresh command line option of regrid_data_plane should be used.  Let's say one target output grid box has 100 GOES-16 pixels inside of it.  The -valid_thresh option defines the ratio (between 0 and 1) of those 100 pixels which must meet the requested -qc flag information in order to get an output value for that grid box.
Sample data can be found on WCOSS, here:
The daily ADP data file can be found in (phase II)
/gpfs/gd3/emc/meso/noscrub/Ho-Chun.Huang/GOES16_AOD/ADP/yyyymmdd
module load NetCDF/4.2/serial
You can use ncdump to read the header, I believe we will read Smoke, Dust, and DQF (for high, medium, and low confidence of Smoke and Dust mask)

ncdump -h OR_ABI-L2-ADPC-M6_G16_s20192472341118_e20192472343491_c20192472344453.nc | more

I discussed the GOES-16 ADP product content with NESDIS/aerosol validation group (Pubu Ciren) on September 18.  The meaning of the Aerosol/Smoke/Dusk mask are different from my previous understanding.
The Aeorosl Mask is NOT an indicator of the AOD retrieval of "ALL" aerosol nor "Aerosol other than Smoke and Dust".
The Aerosol mask = 1 if [Smoke=1 OR Dust=1].  Thus there is no information of any pixel that has aerosol other than smoke and dust (such as anthropocentric aerosol) provided by ADP products.  It is impossible to compute the fraction of Smoke AOD or Dust AOD over total AOD for each mapping grid.
NESDIS GOES-16 ADP product can only identify which ABI (sensor) pixel likely are all smoke/dust particle in its footprint, with confident level determined by the quality flag [qf] (bit 2-3 for smoke and bit 4-5 for dust).  Since Smoke/Dust AOD is the final OBS (what we want) to be compared to same modeled variable, thus it may need to be read in by regrid_data_plane as additional constraint to find the AOD pixel to be gridded to selected model grid.
Note we already have total AOD qf to control the AOD mapping in input GOES-16 ADO file.  This function should be retained because of the need to other broad users out there.  For NCEP operational need as special cases, to find the mapped Smoke AOD in model grid.  We need to add additional condition of
High quality smoke AOD (-qc 0) = AOD with [high AOD qf]+[high Smoke confidence qf], i.e., total AOD qf <= 0  && smoke mask qf <=0.
Medium quality smoke AOD (-qc 1) = AOD with [high AOD qf]+[medium Smoke confidence qf] && [medium AOD qf]+[high Smoke confidence qf] && [medium AOD qf]+[medium Smoke confidence qf]. i.e., total AOD qf <= 1 && smoke mask qf <= 1 ??
And
Low quality Smoke AOD is total AOD qf <= 2 && smoke mask qf <= 2, ??
I do not know the possibility of each pair will occurred in reality, I need you expertise to refine it if you like this approach.
I still want to use current regrid_data_plane function for total AOD comparison to that of NAQFC/CMAQ or FV3GFS-CHEM (or GEFS-aerosol).
For the Smoke AOD mapping I would like to create to be compared to HYSPLIT/HRRR-smoke smoke AOD output
Using -qc 0, -qc 0,1 and -qc 0,1,2.
I attached a new ADP description from NESDIS USer's Guide here
GOES-16 ADP Users Guide extract.pdf
Full version of the users guide can be found in IDL readme docs in John's comment.

The users of Smoke/Dust AOD (for now) probably are NCEP and ESRL (HRRR-smoke group).  The other satellite are VIIRS AOD and ADP.
METplus's algorithm need to be as general as possible to operate on other satellite inputs. We can keep the current option input for regular regrid_data_plane users, but use several environmental settings to tell the algorithm whether it needs additional constraint, where is the input file (i.e., ADP file) , and what is the parameters to be use as a criteria "Smoke mask" or "Dust mask".

When you run regrid_data_plane with AOD input, you can request:

All AOD in the output (name="AOD")
AOD Smoke in the output... this requires ADP to be processed as well (name="AOD_Smoke")
AOD Dust in the output... this requires ADP to be processed as well (name="AOD_Dust")

Need a way to specify the ADP file to be processed, either a command line option on environment variable.
Here are two matching AOD/ADP file names:
/gpfs/gd3/emc/meso/noscrub/Ho-Chun.Huang/GOES16_AOD/ADP/20190919/OR_ABI-L2-ADPC-M6_G16_s20192622351163_e20192622353536_c20192622354379.nc
/gpfs/gd3/emc/meso/noscrub/Ho-Chun.Huang/GOES16_AOD/AOD/20190919/OR_ABI-L2-AODC-M6_G16_s20192622356163_e20192622358536_c20192630000280.nc
The start (s) and end (e) times must match but the creation time (c) will not.
Use the logic Ho-Chun described previously to derive AOD_Smoke or AOD_Dust.

  export MET_GEOSTATIONARY_DATA=/meso2/noscrub/Ho-Chun.Huang/GOES16_AOD/g16_conus_latlon_2km_20180620.dat
  export MET_TMP_DIR=/stmpp2/${USER}/METPLUS_TMP_${mdl_name}/${NOW}
  if [ -d ${MET_TMP_DIR} ]; then
     /bin/rm -f ${MET_TMP_DIR}/*
  else
     mkdir -p ${MET_TMP_DIR}
  fi

case ${i} in
high) value="0"
;;
medium) value="0,1"
;;
low) value="0,1,2"
;;
med_only) value="1"
;;
low_only) value="2"
;;
*) echo "qc_flag=${i} is not defined"
exit;;
esac
aod_file is the GOES-16 AOD file, e.g., files in /meso2/noscrub/Ho-Chun.Huang/GOES16_AOD/AOD/yyyymmdd
mod_file is the model_aot_grib2 file, e.g.,
mod_file=/naqfc/save/${USER}/plot/cmaq/parm/aqm.t12z.aot.f15.148.grib2
output file is the model grid mapped file, e.g., nc file in /meso2/noscrub/${USER}/GOES16_AOD/REGRID/aqm.20190919/OBS_AOD_aqm_g16_20190919_12_high.nc
regrid_data_plane ${aod_file} ${mod_file} ${output_file} -field 'name="AOD"; level="(,)";' -method UW_MEAN -v 1 -qc ${value}

Checked in to feature_1194_adp

Merged to develop branch

The description of the smoke and dust mask confidence level has been updated
(OLD)
; bit 6 (sunglint)
tmp_mask=DQF
idx = WHERE(((tmp_mask AND 64) EQ 64), COMPLEMENT=cidx, nc)
IF nc GT 0 THEN smoke_con[idx] = 0
IF nc GT 0 THEN dust_con[idx] = 0
;Byte1, bit7 (solar and satellite angles)
tmp_mask=DQF
idx = WHERE(((tmp_mask AND 128) EQ 128), COMPLEMENT=cidx, nc)
IF nc GT 0 THEN dust_con[idx] = 0
(NEW and Correct one)
; bit 6 (sunglint)
tmp_mask=DQF
idx = WHERE(((tmp_mask AND 64) EQ 64), COMPLEMENT=cidx, nc)
IF nc GT 0 THEN dust_con[idx] = 0
;Byte1, bit7 (solar and satellite angles)
tmp_mask=DQF
idx = WHERE(((tmp_mask AND 128) EQ 128), COMPLEMENT=cidx, nc)
IF nc GT 0 THEN dust_con[idx] = 0
IF nc GT 0 THEN smoke_con[idx] = 0
Please update  MET9.0 beta version before official release.
I aslo want to point out that from AOD files
AOD high quality is quality flag=0
AOD medium quality is quality flag=0,1
AOD low quality is quality flag=0,1,2
For ADP files
Smoke Mask high confidence flag=3
Smoke Mask medium confidence flag=2
Smoke Mask low confidence flag=1
Dust Mask is the same as Smoke Mask


Updated to Firestore v7.x.x

Fix install grpc failed on node 13
https://github.com/grpc/grpc-node/issues/1216
build grpc issue already fixed in 1.24.2
and firestore^6 force install grpc 1.23.3
firestore upgraded grpc 1.24.2 in 7.3.0

Hi @ClickerMonkey
see if you have time for review the PR please!
thank you!

@CasperLaiTW I was getting the same issue with node 12. Does the firebase bump fix for node 12 also, or only node 13?

@MrDrummer, I used node 12 and 13, all working well.


Update informations for feet for TALOS + python format

[format] Makes flake8 happy.
Uses Parameter server to set:

Remapping of end effector bodies to canonical names: l_ankle, r_ankle, r_wrist, l_wrist to be used by sot-pattern-generator
Specify additional information of feet size on the ground.

@olivier-stasse : should this be included in the release with https://github.com/stack-of-tasks/sot-core/pull/156 ?

And I don't see how this PR "Allows to compile the package without ROS."

@olivier-stasse : should this be included in the release with stack-of-tasks/sot-core#156 ?

Yes please.

And I don't see how this PR "Allows to compile the package without ROS."

Thanks for pointing this out. I fixed the comments.


Validation contact form:

In this PR I added new styles and functionalities like:
-validation if the input on the email is in lowercase or not. (if not refuse the submit action and show a message).

@LionRouge1 is already functioning or still having problems with the servers? I delete it because a reviewer recommend me this


System Status Querying & Pro-active Followup

Build out a REST route/method for getting the status of:

condor status
k8s statuses (server, tms, workers)

If the server dies, abort the scan

#61 gets the statuses of the server and tms/starter

https://github.com/WIPACrepo/SkyDriver/issues/63 covers the pro-active followup


[ISSUE #6511] Correct variable name in MessageStoreConfig

What is the purpose of the change
fix #6511
Brief changelog
XX
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Yes, I have also considered this aspect of the problem. However, we will eventually have to face this problem.
At present, it seems that only the broker side is affected, and there are three solutions for this problem:

Leave it alone and keep it as it is.
Direct modification, not compatible with lower versions, please describe the requested modification in the release document.
Compatible with lower versions, you can add a compatible processing interface to the MixAll.properties2Object(p,o) method, which not only solves the current problem, but also achieves mapping and conversion capabilities.


feature: Update Splat.Autofac to work with Autofac 5.0 (#465)

Raises Autofac version to 5.x in Splat.Autofac, and deprecates or removes method that mutate the container (since that is supposed to be immutable from now on).
Register method is to be used only by the ReactiveUI initialization procedure, which modifies the container by creating a child scope for each service registration.
Thanks to "skaman" for coming up with the workaround.

What kind of change does this PR introduce?

feature: Update Splat.Autofac to work with Autofac 5.x #465
What is the current behavior?

Splat.Autofac is currently locked to at most 4.9.6 version.
What is the new behavior?

Autofac dependency now targets version 5.x
It now (mostly) conforms to Autofac 5 container immutability.
What might this PR break?
Setup procedure is different. Before, everything was done on the IContainer (after the application's container has been built). Now (since containers are not mutable anymore), part of the setup happens earlier, on the ContainerBuilder.
Before:
`/* After the ContainerBuilder has been built */
container.UseAutofacDependencyResolver();
Locator.CurrentMutable.InitializeSplat();
Locator.CurrentMutable.InitializeReactiveUI();`
Now:
`/* While we still have the ContainerBuilder */
// Creates and sets the Autofac resolver as the Locator
var autofacResolver = containerBuilder.UseAutofacDependencyResolver();
// Register the resolver in Autofac so it can be later resolved
containerBuilder.RegisterInstance(autofacResolver);
// Initialize ReactiveUI components
autofacResolver.InitializeReactiveUI();
/* After the ContainerBuilder has been built */
var autofacResolver = container.Resolve<AutofacDependencyResolver>();
// Set a lifetime scope (either the root or any of the child ones) to Autofac resolver
autofacResolver.SetLifetimeScope(container);`
Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

I would appreciate if further testing could be done. I used this with a rather big application I am developing, and nothing broke. This of course does not mean, that no mistakes have been made.
Other information:

If you can update the readme.md file in the autofac directory with the new instructions that would be great.

I'll merge this in AM my time to give time for the CI etc to work, and release later in the week.

I'll try to fix up the tests as well

Thanks

I will add reworked Unregister methods back in, in case somebody really needs them.
And add guards against trying to mutate the ContainerBuilder after an application's container has been built and SetLifetimeScope called.

No problem. Just let me know when you are ready for a full review and merge.

One pitfall about Unregistered. It's a bit hacky, since unregistering a service requires creating a new ContainerBuilder.
That means, one would have to pass the application's ContainerBuilder in like this: new AutofacDependencyResolver(ref containerBuilder).

There's a similar approach with Microsoft di. We don't predict users will use it much and is more for native splat di.

Well, Splat does not need the Unregistered methods, that's why I skipped them at first.

There are cases where's it's needed. Mostly due to InitializeReactiveUI forcing dependencies on users in certain scenarios.

There will be eventually a refactor of the init but it will allow users to work until that happens.

Also there are initialization di hooks that will register containers. Like avalonia uses those for example

Ok then, I'll do my best

Just realized about one big issue. Building containers will cause the following to fire: AutoActivate, Start on Autofac.IStartable and any RegisterBuildCallbacks.
And building the containers is necessary, since that is the only way to get ComponentRegistry of all registered services, which is needed to repopulate a new builder in Unregister methods...

I added the check against trying to register new services to the dependency resolver, once the container has been built.
But I don't have any good idea how to solve the Unregistered methods issue, since firing post build callbacks is not an acceptable side-effect. Could you perhaps point me to where I could find calls of UnregisterCurrent/UnregisterAll ?

You can forgo unregister just make a readme comment about it.

It's only used by users if they don't want a view locator for example rxui uses. The users I know who are using it are on the main splat dictionary DI container.

It's only used by users if they don't want a view locator for example rxui uses. The users I know who are using it are on the main splat dictionary DI container.

I am also using a custom ViewLocator (I am using it to resolve Views in the same scope as its ViewMode with Autofac), and my solution is simple.
Register your custom ViewLocator after InitializeReactiveUI:
autofacResolver.InitializeReactiveUI(); containerBuilder.RegisterType<MyCustomViewLocator>().As<IViewLocator>().SingleInstance();
Since Autofac's order of registration is deterministic, injecting IViewLocator will always return the custom ViewLocator.

Yeah, so it negates the need I guess with autofac so we can just leave those as firing a not-implemented exception.

Yeah, so it negates the need I guess with autofac so we can just leave those as firing a not-implemented exception.

Another way registering custom services (e.g. ViewLocator) could be solved is by calling HasRegistration on this line, similar to how InitializeSplat handles that

If you make a issue in rxui it can be looked at in future versions.


[fine_tuning] Test on the Instructlab cluster

test passed internally, merging (without the WIP PR)
local-ci/rhods/fine_tuning/20241121_1603


Add proto validation for entity type in ProvidersService

Summary
Ref https://github.com/stacklok/minder-stories/issues/94
Change Type
Mark the type of change your PR introduces:

[x] Bug fix (resolves an issue without affecting existing features)
[ ] Feature (adds new functionality without breaking changes)
[ ] Breaking change (may impact existing functionalities or require documentation updates)
[ ] Documentation (updates or additions to documentation)
[ ] Refactoring or test improvements (no bug fixes or new functionality)

Testing
Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.
Review Checklist:

[x] Reviewed my own code for quality and clarity.
[ ] Added comments to complex or tricky code sections.
[ ] Updated any affected documentation.
[ ] Included tests that validate the fix or feature.
[ ] Checked that related changes are merged.

coverage: 54.763%. remained the same
when pulling baae081fb71418e6d1ea292a3e60c8c45d85829f on validate-providers
into 3872633d436f63fda7c3f0cb842258153946f6bb on main.


New location search implementation for droplet parcels

For a particular parcel, the current location search implementation in the droplet_driver subroutine always started from the end of a dimension and scanned backwards until the parcel was found. Such a linear search strategy at every time step was computationally costly and not necessary since a parcel would not move too far.
According to @johnmauff's suggestions, it is better to store the previous location of a parcel and use that as the starting search point for the next time step. In addition, the search range could be restricted to a small value with the assumption that a parcel does not move too far from one time step to another.
This PR introduces the new implementation following @johnmauff's suggestions. We also add a verification test and could confirm that the new implementation returns the identical location indexes as the original implementation.

Thanks for addressing the feedback.


Requires StorageV2 (general purpose v2) storage account

The Events option is only available in StorageV2 (general purpose v2) storage accounts. It's easy enough to upgrade existing accounts, but please mention this.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b250ca39-0ba3-20dc-179b-c5a0d80fba9f
Version Independent ID: 5f1d000e-c731-4b98-8eb7-55836829ca47
Content: Blob storage events for Azure Event Grid with the Azure portal
Content Source: articles/event-grid/blob-event-quickstart-portal.md
Service: event-grid
GitHub Login: @tfitzmac
Microsoft Alias: tomfitz

@MathieuDiepman Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

@MathieuDiepman - thanks for this feedback. I moved some text about Blob and StorageV2 to a place where I think it will be more helpful.


Add support for aarch64-apple-darwin

Did a quick test build for it and seems the main issue is or rpmalloc crate which is lacking support (https://github.com/EmbarkStudios/rpmalloc-rs/issues/11). We could for now disable the custom global allocator for now for this target

Just ran into this. Would you accept a PR that disables that dependency on aarch64-apple-darwin? I can prepare one tomorrow.

We probably should just switch to the mimalloc crate which we already use on on all targets (including aarch64-apple-darwin) in other projects.
A PR with that would be most welcome!

Oh, even easier. I can do that too!

Question though: why does this crate even need a custom allocator? Is it that allocation-heavy that it really benefits much from a non-system one?

I think it was primarily to speed up x86_64-unknown-linux-musl since its allocator was pretty terrible wrt parallelization and it's easier to just use the same allocator for all targets (until we hit issues like this).

Yes I think that was the original issue, and maybe also that on Windows it was really slow with the system allocator that made an actual impact.


Current 3PAR python module does not seem to be compatible with Alletra MP.

We have tested few functionalities with Alletra 9k, but the same is not appearing to be compatible with Alletra MP. It failed from the authentication itself.
Here are the tests and observations:

Tried generating alletra_mp client object via attached python script. Got error about an invalid session_key. Attached snapshot for quick reference.
alletra_mp_auth.py.txt

curl POST requests to WSAPI end points are successful for both Alletra MP and Alletra 9k. Sample run command as below.
$ curl -X POST https://10.XX.XX.53:443/api/v1/credentials -k --insecure -H "Content-Type: application/json" -d  '{"user": "storage_admin", "password": "storage_password"}' -L | python3 -m json.tool

Module versions:
hpe3par-sdk == 1.2.2
python-3parclient == 4.2.14
Can you kindly confirm if the 3PAR python module is tested for Alletra MP or there is another module in development for the same.
We are ready to have quick walk through session of the above if required. Also let us know if we are missing anything. Thanks in advance.
Karthik VR
<EMAIL_ADDRESS>

It looks like Alletra MP requires authentication on the WSAPI version endpoint whereas the others don't.

https://github.com/hpe-storage/python-3parclient/blob/master/hpe3parclient/client.py#L247


Automatic currency conversion

Hello!
When I enter the lei currency (my country's currency), it does not automatically update, by default 0.000000. The System / Settings / Location / Automatic Currency Update setting is YES.

Hello!
Materialize Template does not affect the work with currencies. The solution to this problem should be found in the OpenCart forums.
As far as I know, the old versions of OpenCart used yahoo services to automatically update the currency, now these services are closed. I'm sure there is a solution on the Internet for older versions of OpenCart.

Hello!
Thank you for replying ... I mention that I use OpenCart 3020 ..
Happy Birthday! Good luck!

--


Can't install nuget WebView2.DOM 0.1.3 on Windows 10

Hello:
I want to try your repo. I have done the following:

Create one C# console app with target .net 5.0 on Windows 10 (Version 20H2)
(OS Build 19042.870)
Install webview2:
Install-Package Microsoft.Web.WebView2 -Version 1.0.824-prerelease
Install WebView2.DOM:
Install-Package WebView2.DOM -Version 0.1.3

But I got the following error:
Severity	Code	Description	Project	File	Line	Suppression State
Error	NU1202	Package WebView2.DOM 0.1.3 is not compatible with net5.0 (.NETCoreApp,Version=v5.0). Package WebView2.DOM 0.1.3 supports: net5.0-windows7.0 (.NETCoreApp,Version=v5.0)	WebView2DOMTest	C:\WebView2DOMTest\WebView2DOMTest.csproj
I think there was some configuration not correct in the nuget package, please check and change it.
Thanks,

Hi. This project requires either a WinForms or a WPF project to be installed via NuGet. Console app doesn't work because by default it's cross platform, while the WinForms and WPF projects are Windows only. Hence the “net5.0-windows7.0” version on my package.


Auth codes don't work

When trying to authenticate as an admin using the /auth {code} chat command, an "Incorrect Code" error is returned and no priviledges are assigned.
This behaviour has been seen across multiple browsers and seems to occur for both the randomly generated auth codes, as well as a user-provided auth code.

I can't seem to replicate the error you are receiving. Are you omitting the { } when entering the code in chat?

I am indeed.
I'm launching the Docker container with the following command:
docker run --name=radium -p 3000:3000 -e BASE_URL=http://example.domain -e ADMIN_TOKEN=12345 zibbp/radium:latest

And then authenticating with /auth 12345 as shown in your attached screenshot.
Having looked at the browser console, it appears that you try to connect to the websocket over port 80 which will not work because of both CORS and because I'm running this on port 3000.

Hope this helps!

Appreciate the quick investigation and resolution. Thanks for your help!

Just released v1.1.9 and is pushed to dockerhub.
If you are still having issues, feel free to reopen the issue.

Changes made in v1.1.9 have introduced new issues. Please see console output below.
root@VCinema:~/Radium# docker run --name=radium -p 3000:3000 -e BASE_URL=http://example.domain:3000 -e ADMIN_TOKEN=12345 zibbp/radium:latest

><EMAIL_ADDRESS>start /opt/app
> nuxt start

 ERROR  ServerMiddleware Error: Cannot find module './routes/test'
Require stack:
- /opt/app/api/index.js
-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>- /opt/app/node_modules/nuxt/bin/nuxt.js

  Error: Cannot find module './routes/test'
  Require stack:
  - api/index.js
  -<EMAIL_ADDRESS>  -<EMAIL_ADDRESS>  -<EMAIL_ADDRESS>  - node_modules/nuxt/bin/nuxt.js
  at Object.<anonymous> (api/index.js:8:14)
  at Generator.next (<anonymous>)

Pushed another update v1.1.10  fixing that error. I had forgot to remove the import for the test route
I should set up automatic testing so situations like this don't happen again.
Sorry about this.


Scaling depending on the printer model

Some printers generate really thick bars in the barcode, ex:
 let point_width = context.barcode.width = 10;
 let point_height = context.barcode.height = 64;

In that case, the barcodes become unreadable because of the thickness of the bars. I tried changing the scale, but it didn't work. Do you have any clue about the solution for this?
OBS:  it happens for all barcodes generated for this specific printer. In order to reproduce it, you can just set the width to 10 and heigth to 64 harcoded.

Can you send me the binary for that barcode command? I assumed the width is the pixel width of each bar but maybe I'm wrong on the implementation.

Here is the link with the file: https://drive.google.com/file/d/1tT80VIXjlsVaXCeFIkBwkUv7kimcgXZJ/view?usp=sharing, cant send files here.

Also, just noticed it, it's cutting part of the barcode.

Looking through the spec, it seems like GS w is in fact the width of each bar, so a 10 would be really thick.
https://www.epson-biz.com/modules/ref_escpos/index.php?content_id=129
I could add some kind of override option into the context like force_single_barcode_width which would force all barcode width settings to use 1 instead of the context width.
There is this other line in the spec which I don't quite understand that may be related:
When the barcode height set with GS h is smaller than [33 times the module width], a barcode with a height (excluding the HRI characters) of [module width × 33] is printed, without reference to the GS h setting.

I think we can limit the min to 2 and the max to 6, and divide the number in half for the pixel width.
w   value     px
1       2          1
10     6         3

What do you mean by “The width value that gets set on the context is divided by 2”? I didn't get the point.

I'll see if I can implement and test to see if my idea works. It's a but hard to explain until I can get back into the code.
The 2 and 6 limits come from this: https://python-escpos.readthedocs.io/en/latest/user/barcode.html