How to write data with struct with pyarrow.parquet ?

I want to convert a JSON data file to parquet format.
The only API to write data to parquet is write_table(). But I find no way to create a table with structs, either by table.from_array() or table.from_pandas(), but I always got errors.
For example:
data = [
   pa.array([1,2,3]),
   pa.array({"foo":"bar"}, {"foo":"bar"}, {"foo":"bar"})
]

TypeError: Argument 'type' has incorrect type (expected pyarrow.lib.DataType, got dict)

or
df = pd.read_json('[{"foo":{"foofoo":"bar"}}, {"foo":"barbar"}]')
table = pa.Table.from_pandas(df)

pyarrow.lib.ArrowInvalid: Error inferring Arrow type for Python object array. Got Python object of type dict but can only handle these types: string, bool, float, int, date, time, decimal, list, array

Hi, we do not yet support the syntax you are using for creating an array of structs. After adding missing square brackets:
In [2]: pa.array([{"foo":"bar"}, {"foo":"bar"}, {"foo":"bar"}])
---------------------------------------------------------------------------
ArrowNotImplementedError                  Traceback (most recent call last)
<ipython-input-2-5383db65d4fd> in <module>()
----> 1 pa.array([{"foo":"bar"}, {"foo":"bar"}, {"foo":"bar"}])

/home/wesm/code/arrow/python/pyarrow/array.pxi in pyarrow.lib.array (/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:25024)()

/home/wesm/code/arrow/python/pyarrow/array.pxi in pyarrow.lib._sequence_to_array (/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:23628)()

/home/wesm/code/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status (/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:8242)()

ArrowNotImplementedError: /home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:740 code: InferArrowTypeAndSize(obj, &size, &type)
/home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:319 code: InferArrowType(obj, out_type)
/home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:299 code: seq_visitor.Visit(obj)
/home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:180 code: VisitElem(ref, level)
No type inference for dicts

I would like to support this, but implementing it is quite some work. Patches are welcome. I opened https://issues.apache.org/jira/browse/ARROW-1705 to track the feature request.

By the way, you can use pa.StructArray.from_arrays to create a struct:
In [3]: fields = [pa.array(['foo', 'foo', 'foo']), pa.array(['bar', 'bar', 'bar'])]

In [4]: arr = pa.StructArray.from_arrays(['key', 'value'], fields)

In [5]: arr
Out[5]:
<pyarrow.lib.StructArray object at 0x7f874084b098>
[
  {'value': 'bar', 'key': 'foo'},
  {'value': 'bar', 'key': 'foo'},
  {'value': 'bar', 'key': 'foo'}
]

Thank you for your correction and info, Wes.
I am not quite sure how to compose nested objects and arrays in pyarrow with some low level primitives, e.g. part of my json fields look like
{"obj": {"foo":"bar"}, "arr": [ {"key":"value1"},{"key":"value2"} ] }
{"obj": {"foo":"bar2"}, "arr": [ {"key":"value3"},{"key":"value4"} ] }
{"obj": {"foo":"bar3"}, "arr": [ {"key":"value5"},{"key":"value6"} ] }

I understand it needs some works and in the future this may be natively supported by pyarrow. For the moment I'd just convert them as JSON strings or flatten them in a obj_foo name style.
Thank you all the same for the great job.

You need to unravel this data into its columnar representation. What's the effective schema here?
obj: struct<foo: string>
arr: list<item: struct<key: string>>

This data has 2 leaf nodes: obj.foo and obj.arr.item.key. So you need to "shred" the leaf data and assemble the Arrow arrays by following the columnar data model:
foo_data = pa.array(['bar', 'bar2', 'bar3'])
arr_data = pa.array(['value1', 'value2', 'value3', 'value4', 'value5', 'value6'])

obj = pa.StructArray.from_arrays(['foo'], [foo_data])

arr_offsets = pa.array([0, 2, 4, 6], type='int32')
arr = pa.ListArray.from_arrays(arr_offsets,
                               pa.StructArray.from_arrays(['key'], [arr_data]))

Now we have:
In [17]: obj
Out[17]:
<pyarrow.lib.StructArray object at 0x7f4ab64daa98>
[
  {'foo': 'bar'},
  {'foo': 'bar2'},
  {'foo': 'bar3'}
]

In [18]: arr
Out[18]:
<pyarrow.lib.ListArray object at 0x7f4ab64e7a98>
[
  [{'key': 'value1'},
   {'key': 'value2'}],
  [{'key': 'value3'},
   {'key': 'value4'}],
  [{'key': 'value5'},
   {'key': 'value6'}]
]

Hope this helps

I don't believe the original answer in this issue was answered - is it possible to write structured data to Parquet? I tried the following:
data = pa.StructArray.from_arrays([pa.array([1])], ['a'])
table = pa.Table.from_pydict({'data': data})
pq.write_table(table, 'test.parquet')

and it gives the error: pyarrow.lib.ArrowNotImplementedError: Level generation for Struct not supported yet.

Not at the moment: https://issues.apache.org/jira/browse/ARROW-1644?filter=-1 is tracking the feature.

As @wesm is mentioned above, I wrote the following code in python:
fields = [pa.array(['foo', 'foo', 'foo']), pa.array(['bar', 'bar', 'bar'])]
arr = pa.StructArray.from_arrays(['key', 'value'], fields)
print(arr)
and I get the following error:
expected bytes, pyarrow.lib.StringArray found
why?

you probably want print(str(arr))

The error is related to "arr = pa.StructArray.from_arrays(['key', 'value'], fields)".

It looks like you have the args reversed: https://arrow.apache.org/docs/python/generated/pyarrow.StructArray.html#pyarrow.StructArray.from_arrays

I reversed that args i.e., "arr = pa.StructArray.from_arrays(fields, ['key', 'value'])" and I got the below result:
-- is_valid: all not null
-- child 0 type: string
  [
    "foo",
    "foo",
    "foo"
  ]
-- child 1 type: string
  [
    "bar",
    "bar",
    "bar"
  ]```


Fix horizontal scroll

Im using your amazing extension in Vivaldi's web panel. It works well, unlike others tab managers. But there is a small glitch: horizontal scroll. Fix is easy: set overflow-x: hidden. Or something like that.

Thanks for the feedback @averrin!
Do you mean the horizontal scroll doesn't work on the https://vivaldi.com/ browser?

I can't reproduce this in Vivaldi

Sorry for long answer. I specifically said about "web panel". It isn't regular tab, its smaller docking window in left panel.

And it has a horizontal scroll in every panel width.

Got it, thanks. I will double-check in 1-2 days.

Confirmed I could reproduce the same issue, will have a fix shortly.

0b4695e5f336e106f5947d532751a1f776c44b78 the fix commit

@averrin it should be fixed in v1.15.2, could you please confirm?


updated data url, fixed conform

Merging because data build was successful even though machine reported an error.

d7cff92d22fb2b9a926987eeb6d9ba2416dc6fdf


feature(ssh): Make ssh a dedicated feature

feature(ssh): Make ssh a dedicated features
How to categorize this PR?

/kind enhancement
/area os
/os garden-linux
What this PR does / why we need it:
Make ssh & fail2ban a dedicated feature

New feature: SSH
New feature: Fail2ban
Code cleanup (deprecated files, content, comments,...)

Which issue(s) this PR fixes:
Fixes #1193
Fixes #302
Release note:

- category: feature
- target_group: operator

Validating the functionality of fail2ban:
You can see the entry within the chain f2b-sshd:
root@garden:~# iptables -vnL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    2    88 f2b-sshd   tcp  --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/0            multiport dports 22

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain f2b-sshd (1 references)
 pkts bytes target     prot opt in     out     source               destination
    2    88 REJECT     all  --  *      *       <IP_ADDRESS>             <IP_ADDRESS>/0            reject-with icmp-port-unreachable
    0     0 RETURN     all  --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/0
root@garden:~#

This features are tested within the following constellations:

gardenlinux without ssh-server
gardenlinux with ssh-server
gardenlinux with ssh-server and fail2ban for ssh

After merge it is in the operators decision to build the needed images with combinations of ssh and fail2ban or just without anything of this new features which makes Garden Linux more modular based on its feature's system. In default, the server features includes ssh which includes fail2ban. Within a default installation (without any further changes) a ssh-server will still be present including fail2ban for ssh services.
CC: @nanory @gehoern

Hi @gyptazy, this PR will not be part of the next release (within the next few days).
Additionally, the feature ssh and fail2ban should be one feature.
Fail2ban will require some testing, also in gardener context.

Hey @Vincinator,
thanks for the update. For merging fail2ban into ssh we should have a small talk within our tomorrow's weekly meeting. I'd like to avoid this and keep them separated.
Thanks!
See you tomorrow,
gyptazy

Yes, makes sense to sync in a weekly.
I am not invested in either decision. I was the messenger :)
And the key message is: Do not merge before the next release.
But here is my perspective nonetheless:
Pro: merge into one feature:

design decision is to not replicate a packet manager, but to add a higher level of opinionated functionality for the user
fail2ban adds another layer to prevent brute force attacks

Pro: separate features:

would help in cases where Gardener users have to (cleanly) turn it off

Based on the notes from our weekly meeting a new PR has been created. #1351


Error in macros/schema_tests/string_matching/*

All the methods in macros/schema_tests/string_matching/ define a group_by_columns=group_by functionality, but group_by is not defined in the method which makes them fail with 'MacroGenerator' object is not iterable.
Fix should be to set up group_by=None in methods' definition as per the other macros with this functionality.
e.g.,
 {% test expect_column_value_lengths_to_equal(model, column_name,
value,
group_by=None,
row_condition=None
) %}

Hi @samantha-guerriero-cko , thanks for raising that issue! You're right, I see the missing params. What's weird is that this passes our integration tests without an error. For example, this test gets called here.
Could you share some more information about your environment (which platform, what version dbt & Python)?
Also, let me know if you're interested in submitting a PR for this before I jump on this myself.

Btw, I think the correct treatment here is probably to not support group_by for this particular test, since the expression is not an aggregation so it would fail on the group by anyway.
So, I'd set the group_by_columns param to None in the call to expression_is_true:
{{ dbt_expectations.expression_is_true(model,
                                        expression=expression,
                                        group_by_columns=None,
                                        row_condition=row_condition
                                        )
                                        }}

Btw, I think the correct treatment here is probably to not support group_by for this particular test, since the expression is not an aggregation so it would fail on the group by anyway.
So, I'd set the group_by_columns param to None in the call to expression_is_true:
{{ dbt_expectations.expression_is_true(model,
                                        expression=expression,
                                        group_by_columns=None,
                                        row_condition=row_condition
                                        )
                                        }}

It makes sense to make the group_by_columns=None :)
For what concerns my environment:

dbt = 0.21.0 (have tried also with 0.20.0)
Python = 3.9.6
Platform = macOS Big Sur 11.6

I am happy to open a PR for this 💪

@samantha-guerriero-cko amazing, thank you!


Transfer to fanuc-oss organisation?

Something to consider: create an onofficial "Fanuc Open-Source Software" GH organisation and transfer this repository to that organisation, together with ktransw_pi and potentially other Fanuc-specific open-source tools to focus efforts and improved visibility of related projects.
Opening this here to check whether this is something that other Fanuc OSS tool maintainers/developers would be interested in.

@kobbled, @unreal: opinions / comments?

@unreal: I'm specifically curious to know whether you'd be interested in hosting your (awesome) tools like tp_plus, go-fanuc, KUnit, vim-tp and others in such an org, or whether you'd rather keep them on the OneRobotics org.
Right now I have the impression that there are probably quite a few Fanuc-focused OSS projects on GH, but living on personal accounts they are hard to find. Hosting them in a single organisation improves their chances of being found and chances of related / companion projects being found.

I agree it's probably hard to find some of these projects, though I think GitHub's tagging feature (e.g. #fanuc) helps.
Perhaps a better solution with less overhead is a curated list of projects e.g. Awsome Vue.js, Awesome Go, etc.

Awesome lists are nice, but the trouble with them would be that they would have to be maintained by someone.
Hosting related projects in a single org does that sort of automatically.
The tagging certainly helps.

Currently I am just checking the fanuc topic for new projects like @unreal suggested. An org might be a good idea as it might incentivize sharing and open sourcing from a community that isn't accustom to these practices. However, that's more of a philosophical debate on the typical fanuc integrator/developer.

@gavanderhoorn, would you include ros-industrial fanuc related projects under this banner,or is that outside of this scope?

@kobbled wrote:

An org might be a good idea as it might incentivize sharing and open sourcing from a community that isn't accustom to these practices.

yes, that was my thinking as well.

would you include ros-industrial fanuc related projects under this banner,or is that outside of this scope?

I'm not sure, because those packages are not "stand-alone" but allow ROS to integrate/communicate with Fanuc robots. Outside of the ROS context they are not really useful (or usable even).
The tools/utilities/scripts/etc I'm thinking of would be purely Fanuc-specific and not tied to any framework or platform specifically.

Inviting @iBaff and @alek5k to the discussion.

I'm mainly thinking about post-processors. People probably don't use Victor Lamoine's fanuc post processor outside of ros context, but you definitely can.

It's not really a post-processor imo, but I get your point.
(btw: there is a whole bunch of post-processors over at ros-industrial/robodk_postprocessors)

@VictorLamoine: opinion?

Note btw that moving to an org instead of separate user accounts does not imply any loss of control or handing repositories "over" to me or anyone else. All authors would remain in full control of the work of course.
And I also feel there would still be plenty of opportunity to do any branding as desired.

Creating an organization will only let you list a GitHub repos.

True.

I would go for an "Awesome" list.

That would require maintenance, as I wrote in https://github.com/gavanderhoorn/rossum/issues/30#issuecomment-485878880.

Thanks for all your input. I'm going to close this issue, as I was merely curious to see whether there would be enough momentum to come together and form some sort of community around the idea of OSS for Fanuc robots and related applications.
It seems there is something, but maintaining a list/index/overview requires both additional effort and would introduce yet another repository on a personal account.
Perhaps someone else is motivated enough to start an Awesome Fanuc OSS list. In that case, they're more than welcome to put any of my repositories on it :).

@gavanderhoorn I've got more than enough to deal with so I'll pass :smile:
I'll add fanuc tags to my repositories, this will improve visibility.


Fix assertion error when closing / shutdown native channel and SO_LIN…

…GER is set.
Motivation:
When SO_LINGER is used we run doClose() on the GlobalEventExecutor by default so we need to ensure we schedule all code that needs to be run on the EventLoop on the EventLoop in doClose. Beside this there are also threading issues when calling shutdownOutput(...)
Modifications:

Schedule removal from EventLoop to the EventLoop
Correctly handle shutdownOutput and shutdown in respect with threading-model
Add unit tests

Result:
Fixes [#7159].

Cherry-picked into 4.0 (dec1679c6e)


Turn tags on by default and remove setting to turn it off

We're getting close to the point where tags can leave preview and we can remove the setting to disable it. We should hold off on this a bit longer but I'm opening this issue to keep track.

I take it

Before this I believe we need a way to:

allow the user to hide the "Tags" column from all folders at once
allow the user to edit tags as #9710 also removes the "edit tags" command


Segmentation fault when calling mrb_define_method with a NULL class.

I know I'm a little off the beaten trail here with the example, but this segfault should affect all uses of mrb_define_method.
I do not call mrb_init_array which means mrb->array_class is NULL.
This call in mruby-pack segfaults because the class passed to mrb_define_method is NULL.
 mrb_define_method(mrb, mrb->array_class, "pack", mrb_pack_pack, MRB_ARGS_REQ(1));

From LLDB:
Process 55959 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
    frame #0: 0x0000000100213700 airb`mrb_define_method_raw + 32
airb`mrb_define_method_raw:
->  0x100213700 <+32>: movl   0x10(%rax), %ecx
    0x100213703 <+35>: shrl   $0xb, %ecx
    0x100213706 <+38>: andl   $0x80000, %ecx            ; imm = 0x80000
    0x10021370c <+44>: cmpl   $0x0, %ecx
Target 0: (airb) stopped.
(lldb) frame select 3
frame #3: 0x00000001001f49e6 airb`mrb_mruby_pack_gem_init(mrb=&0x101800400) at pack.c:1332:3
   1329	  check_little_endian();
   1330	  make_base64_dec_tab();
   1331
-> 1332	  mrb_define_method(mrb, mrb->array_class, "pack", mrb_pack_pack, MRB_ARGS_REQ(1));
   1333	  mrb_define_method(mrb, mrb->string_class, "unpack", mrb_pack_unpack, MRB_ARGS_REQ(1));
   1334	  mrb_define_method(mrb, mrb->string_class, "unpack1", mrb_pack_unpack1, MRB_ARGS_REQ(1));
   1335	}
(lldb) p mrb->array_class
(RClass *) $0 = &0x0

Hmm, the short answer is call mrb_init_array first, but there must be some reason behind. Could you elaborate?

I think this is a fundamental limitation of C. Uninitialized memory is something I need to deal with 😢
One change I did make which might be upstreamable is to change this line in mruby-pack
https://github.com/mruby/mruby/blob/c33100c4f03b5f0e8d9034a7b82fabb0975e9a5a/mrbgems/mruby-pack/src/pack.c#L1328
to call:
mrb_class_get(mrb, "Array")

so that it doesn't need to rely on how the core is initialized.

There are many places that use mrb->array_class for performance reason. I don't think we can modify them all.

Maybe you can overwrite the mrb->array_class like
https://github.com/mruby/mruby/blob/c20dda4f4c72f2da10194a417bbca6cf6f5457c2/src/array.c#L1314
before initializing the pack gem.

Did you solve the issue, or just abandon?

I figured out a work around. Modifying the semantics of the class fields on the mrb_state is not something I think is a good idea. I've kept the initialization of these fields.

Nice. Thank you.


Implements file://*.zip and dropbox:// schemes

Codecov Report

Merging #42 into master will decrease coverage by 0.48%.
The diff coverage is 73.68%.

@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
- Coverage   79.54%   79.05%   -0.49%
==========================================
  Files           1        1
  Lines         176      191      +15
==========================================
+ Hits          140      151      +11
- Misses         36       40       +4

Impacted Files
Coverage Δ

sibyl.sh
79.05% <73.68%> (-0.49%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f5411d...cb4ec40. Read the comment docs.


Only update window title if necessary

Hi,
On Ubuntu 16.04 (with the kde desktop environment from the kde backports ppa), moving the camera makes the taskbar freeze for several minutes (while using 100% CPU).
This happens because :
When the user does move/rotate the camera
-> the pitch/yaw/... properties are updated
-> setWindowModified( true ) is called to notify the qwidget some modifications are pending
-> in setWindowModified() Qt will update the title bar of the rviz window, to add the ' * '
The problem is that kde's task manager has a hard time processing "new window title" events at several hundred times per second, thus freezes.
This pull request fixes the issue on my system

@ros-pull-request-builder retest this please

@ros-pull-request-builder retest this please


Add guide for installing MiniKF on GCP

Add guide for deploying Kubeflow on Google Cloud Platform using MiniKF,
which is available on the Google Cloud Marketplace.

This change is

/assign

Preview:

https://deploy-preview-1460--competent-brattain-de2d6d.netlify.com/docs/gke/deploy/minikf-gcp/
https://deploy-preview-1460--competent-brattain-de2d6d.netlify.com/docs/started/getting-started/

@sarahmaddox thank you for your comments. I made all the necessary changes and updated my PR.

New preview link: https://deploy-preview-1460--competent-brattain-de2d6d.netlify.com/docs/started/workstation/minikf-gcp/

Thanks! I made one small change, to update the link to the new page since we moved the file.
/lgtm
/approve


HDDS-8276. EC file checksum calculation fails with topology-aware read enabled

What changes were proposed in this pull request?
ECFileChecksumHelper#getChunkInfos copies the original EC pipeline and limits nodes to specific replicas.  However, the list of "nodes in order", used for topology-aware read, still contains all original nodes.  Thus it may happen that read is attempted from the wrong node.
This patch updates "nodes in order" in the copy to reflect the same set of nodes as the unordered one.
Also remove a stray placeholder from the error message:
Fail to get block checksum for ... checksum combine mode : {}COMPOSITE_CRC
https://issues.apache.org/jira/browse/HDDS-8276
How was this patch tested?
Updated the integration test TestOzoneFileChecksum to cover both topology-aware and regular read.
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4553955626

Thanks @sodonnel for the review.


runtime/debug: FreeOSMemory() not working on Android 9

What version of Go are you using (go version)?
GO 1.12 - 1.14
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
On android 9, on other all works
What did you do?
func main() {
	bigArr := make([]byte, 250*1024*1024)
	for i := 0; i < len(bigArr); i++ {
		bigArr[i] = byte(i)
	}
	bigArr = nil
	runtime.GC()
	debug.FreeOSMemory()
	for true {
		//Some thing works in program
		time.Sleep(time.Second)
	}
}

What did you expect to see?
I expected to see the memory in the system, but it remains in the program. If you allocate memory several times, then the program is killed by the system, but should clear it
In version 1.11.X all works in android 9, memory returning to os, but not execute on android 10. On android 5,6,7,8 all works of new golang version
Compile cmd: GOARM=7 GOOS=linux GOARCH=arm go build -o test main

Thank you for the report @YouROK and welcome to the Go project!
/cc @mknyszek @aclements @eliasnaur @cherrymui

Considering the combination of Android version and Go version here (Android 10+, Go 1.12+), I suspect this is another case of MADV_FREE not updating the kernel's per-process RSS numbers. Android 10's minimum Linux kernel version is 4.9 whereas Android 9's minimum is 4.4. MADV_FREE was introduced in Linux 4.5.
Sorry for the very late reply on this. Could you try running your code with GODEBUG=madvdontneed=1? I'm not sure how difficult that is to do on Android.

@mknyszek It is great we can use GODEBUG flags from environment variables. But it would be alto great to be able to hardcode that into the code, or define, depending on the build parameters, which is needed, if you cannot control how the binary is ran by user.

Filed #40870

It's really disappointing if Android kills applications for memory use without first trying to clean up their freed pages... If that is the case, then I would consider MADV_FREE quite broken on Android and we shouldn't use it.
@YouROK, your example code is likely working as expected, but you mention "If you allocate memory several times, then the program is killed by the system". That sounds like the real issue here. Could you post code that reproduces that?

Ping @YouROK . Could you try the GODEBUG=madvdontneed=1 environment variable? And can you post the code that demonstrates "allocating memory several times"?

Re-opening this issue.

I think gopherbot actually did the right thing here. We haven't heard back from @YouROK, so there's not anything more we can do here, and whatever the details of this issue are, it's probably fixed by CL 267100. So I'm going to re-close this issue, but feel free to reopen it with more information, @YouROK .

@aclements with flag GODEBUG=madvdontneed=1 all work good.
I have been looking for the problem on different devices for several months.
I can say for sure that the problem exists on almost all Linux devices, but the memory grows only up to a certain point, which is not critical on a regular PC with Linux. But on android, Linux and Android builds behave the same way, and when a certain point is reached, the Android OS closes the application.
I can suggest one solution to enable the build flag with GODEBUG=madvdontneed=1 to not add it at startup in env

@gopherbot add OS-Android

@YouROK, Go 1.16 defaults to GODEBUG=madvdontneed=1, so setting that environment in Go 1.16 should not have any effect.


fix doc

Small fix, but help the next person!

hey @epugh , did inserting a integer id work out for you? (mysql? mariaDB?)
I recently changed id to varchar and (you're right pointing it out ;-) ) forgot about the readme here, but I assume, a string id is needed ...

Yep, adding the id field in the command made it alll work nicely!
On Fri, Jun 21, 2019 at 1:08 PM pbartusch<EMAIL_ADDRESS>wrote:

hey @epugh https://github.com/epugh , did inserting a integer id work
out for you? (mysql? mariaDB?)
I recently changed id to varchar and (you're right pointing it out ;-) )
forgot about the readme here, but I assume, a string id is needed ...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pbartusch/smui/pull/20?email_source=notifications&email_token=AAAFO6ZFLPDTQS7ONYDIWR3P3TADVA5CNFSM4H2OSHT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYIJENQ#issuecomment-504402486,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAFO66W3MQGBLXSQV6WOKTP3TADVANCNFSM4H2OSHTQ
.

hi @epugh , I updated the documentation. In fact, there already where the REST interfaces, they just needed to be included in the readme and are much nicer than the direct INSERTs.


spanish version for  4.0.6

Hi, do you have any plans for the translation of the new version?
Regards,

Sorry, it's done. Updated to 4.0.7 + several fixes/typos.


feat: Adds theming & ✨ fix: color issues, layout and getAuth functions

feat: Adds theming & ✨
fix: Color issues, layout, and getAuth functions.
This PR introduces the following changes to ensure the IQ.wiki platform's design is aligned with its branding:

Updated Tailwind config to apply IQ.wiki’s primary and secondary colors.
Added toasts using the Sonner package for consistent notifications.
Integrated the Everipedia/login package for authentication.
Fixed color inconsistencies and improved layout across components to match IQ.wiki's design.
Ensured dark mode compatibility throughout the platform.

Closes https://github.com/Adebesin-Cell/IQ.vote/issues/1

Yolo 🚀


Add start to Note constructor as an alternative to delta

We agreed that delta could be relegated to internal use only, with users being encouraged to use absolute time attribute Note.start and Note.end. How does this work when people create Note objects? Currently the way this works is that users create Note objects specifying delta and then later call parent.insert on those notes. What would the equivalent be if users create Note objects specifying start instead? If I create a Note object with start=4, and then insert it into a Measure which also has start=4, does the Note end up with delta=0 or delta=4? Presumably start should be interpreted as an absolute time, and so delta ends up being 0? Need to check that this doesn't raise any complications down the line.

I think one such complication is working out how insert can know whether a Note was created in absolute time or relative time. Currently Note(start=4) and Note(delta=4) would create identical objects (with parent=None and delta=4) and so insert would not know that it is meant to interpret the former in absolute time.

Except delta has meaning as beat position within the measure, which is used to advantage at least in some machine learning algorithms I'm familiar with. Given that and your example, I think delta is more natural for Note construction. If users really want to avoid thinking in terms of position-within-measure, maybe they should construct a flattened score and we should provide a function that inserts measures. In that case, delta is still the right thing to pass to the constructor, but since the parent start is zero, the Note will have start=delta.

I think my issues could be addressed if we made parent part of the __init__ signature, so that we know from the outset what delta is referring to. Would you be happy with this? It would necessitate a bit of refactoring (and maybe I hit unforseen issues with this) but I think it should protect us from some of these difficulties (e.g. Note.start delivering incorrect values until insert is called.


Is EveryFixedUpdate called in same frame or next frame?

I couldn't find this in the docs, either.  Thanks for any insight.

Where do you start subscrption?
In update? In coroutine-update? In fixed-update? In last-update?
But FixedUpdate's lifecycle is different with standard update.
It is undefined.

Sometimes I start it via a subscription to Observable.EveryFixedUpdate, other times from a subscription to Observable.EveryUpdate.  Is its behavior deterministic for each of those two cases?  If so, what is the expected behavior?

EveryUpdate/EveryFixedUpdate follows WaitForFixedUpdate
https://docs.unity3d.com/Manual/ExecutionOrder.html
The frequency of FixedUpdate differs according to the setting, and it follows the setting of Unity.

Thanks!  That is exactly what I needed to know.


Add custom event support

Adds assigning listener syntax:
let click = Callback::from(|_| ());
html! {
    <div
        on:{click}
        on:click={Callback::from(|_| ())
    />
}

If / when we support adding event listeners onto Components directly it will distinguish
when an event listener is being assigned to a Component instead of being passed down as a prop:
// WON'T COMPILE TODAY (for anyone skimming through looking at examples)

// pretend I defined `click` and `onchange` correctly :)
<MyComp
    // event handler being used on component
    on:{click}
    // callback being passed down to the component for internal use
    {onchange}
/>

Add custom event support
Custom event support comes from implementing the StaticEvent trait:
pub trait StaticEvent {

   type Event: JsCast + AsRef<web_sys::Event> + 'static;

   fn event_name() -> &'static str;
}

On it's own this would allow a user implement the trait with the Event
type being the wrapped web_sys event.
struct MyCustomEvent(web_sys::Event);

impl StaticEvent for MyCustomEvent {
    type Event = web_sys::Event;

    fn event_name() -> &'static str {
        "custom"
    }
}

html! {
    <div
        // here callback gets a web_sys::Event and not the MyCustomEvent type
        // but the user could just wrap the type MyCustomEvent(e) then call
        // any useful functions. Not bad...but we can do better :)
        on:MyCustomEvent={Callback::from(|e: web_sys::Event| ())}
    />
}

A similar pattern is used to implement the events supported by yew, but with unit based structs as we just want the raw web_sys event passed in.
This allows us to treat supported and custom events in the same way and reduces the need for multiple on[event] modules.
This also means that if Yew wanted to provide it's own events somewhen then we could easily utilise the custom_event macro to make NewTypes around the raw web_sys events and this would work the same internally.
When implementing the trait like this if a user wanted to use a function on the NewType they would
have to wrap the event type in that NewType first, which doesn't seem very ergonomic.
It is possible for a user to use Self as the Event type but then they have
to implement JsCast on that type, which is not trivial even when wrapping a JsCast type.
To help with the boiler plate, this PR also adds the custom_event attribute macro which
can be applied to NewTypes that wrap a type that implements JsCast + AsRef<web_sys::Event>, this includes any
type imported in a wasm_bindgen extern block that extends Event.
The custom_event macro has one attribute which is used to define the "event name"
if this name is representable in Rust then a shorthand can be used:
Shorthand
// bind ident to event name `custard`
#[custom_event(custard)]
// Newtype here

Normal
#[custom_event(bizarre_event = "    represent ''' this in Rust")]
// Newtype here

The ident defined in the attribute can then be used in the same way as other events:
// each callback would be accepting the NewType that the custom_event macro
// was used on!

// can still use the shorthand in html too if the variable matches!
let bizarre_event = Callback::from(|_| ());
html! {
    <div
        on:custard={Callback::from(|_| ())}
        on:{bizarre_event}
    />
}

Putting it all together:
#[custom_event(custom)]
struct MyCustomEvent(web_sys::Event);

// The shorthand works here when the variable name matches the type name
html! {
    let custom = Callback::from(|e: MyCustomEvent| ());
    <div
        on:{custom}
        // you can still use the struct name if you prefer!
        on:MyCustomEvent={Callback::from(|e: MyCustomEvent| ())}
    />
}

When implementing the static_event_impl macro I added optional comments and they display really well in the html macro for VSCode, I've only added a comment about the formdata event:

This could be done as is with the modules but I think it's a nice touch so have added it here as I was using a new macro_rules macro anyways.
I have changed the syntax and implementation a few times over getting to this point
but I don't think it's in bad shape (hopefully ;P) and look forward to hearing what
people think :)
Typed Custom Event
I think it's worth just going over the example that triggered the linked issue and how a solution
might look with this change. It was the ability to listen to the "MDCSnackbar:closing" event and others like it on
imported web components (in the example the mwc-snackbar).
How this example looks today:
Snackbar in material-yew
A possible solution with this PR:

use wasm_bindgen::prelude::*;

// DetailsReason implements JsCast when imported in by wasm_bindgen
#[wasm_bindgen]
extern "C" {
    #[derive(Debug)]
    #[wasm_bindgen(extends = Object)]
    type DetailsReason;

    #[wasm_bindgen(method, getter)]
    fn reason(this: &DetailsReason) -> Option<String>;
}

use yew::custom_event;

#[custom_event(snackbarclosing = "MDCSnackbar:closing")]
struct SnackBarClosingEvent(web_sys::CustomEvent);

impl SnackBarClosingEvent {
    // web_sys::CustomEven` has a `detail` method which returns a JsValue, which is not very
    // user friendly. As we have wrapped the CustomEvent in our new type we can now define our own
    // detail method that returns our imported type.
    pub fn detail(&self) -> DetailsReason {
        // JsCast::unchecked_into to jump from JsValue to DetailsReason
        self.0.detail().unchecked_into()
    }

    // You may just want to skip the detail call and provide a direct method to the reason
    pub fn reason(&self) -> Option<String> {
        self.0.detail().unchecked_into::<DetailsReason>().reason()
    }
}

// Then inside the SnackBar Component a Callback can be used like any other supported event

use yew::prelude::*;

#[derive(Properties)]
pub struct SnackBarProps {
    #[prop_or_default]
    pub onclosing: Callback<Option<String>>,
    // rest omitted for clarity
}

impl Component for MatSnackBar {

    // create, update, etc omitted for clarity

    fn view(&self) -> Html {
        let snackbarclosing = self.props.onclosing.reform(|e: SnackBarClosingEvent| {
            e.detail().reason()
        });
        html! {
            <mwc-snackbar
                on:{snackbarclosing}
                // rest omitted
            >
                { self.props.children.clone() }
            </mwc-snackbar>
        }
    }
}

This Component could make the SnackBarClosingEvent and it's snackbarclosing alias public and
change the onclosing property to accept this event. This approach means the event can be defined
with the Component and imported by the user :)

Fixes #1777
Checklist

[x] I have run cargo make pr-flow
[ ] I have reviewed my own code
[x] I have added tests

Why close this?

Why close this?

Didn't seem to get much interest and the on: listener syntax is probably a touch pre-emptive before (#1533 / #1779) is resolved.
#1542 added ListenerKind enum which blocks being able to register custom events using &'static str, I could add some other variant which takes the str or some other method for registering a custom listener but wasn't sure it was worth keeping this approach going.
Also as you suggested in the linked issue an API on NodeRef might be a better alternative considering the above and would probably require far less macro magic and wouldn't need the transmute that I sneaked in here :D


Empty PR to test ASAN bot

Please test with following pull request:
https://github.com/apple/swift-driver/pull/948
@swift-ci Please test ASAN

Please test with following pull request:
https://github.com/apple/swift-driver/pull/948
@swift-ci Please ASAN test

Interpreter/SDK/weak_objc_interop.swift failed, but this seems a transient failure that sometimes happens in ASAN bot.


Fix crash when USB device numbers reach 256

Currently, the Smart Card Connector crashes once it receives a USB device number from Chrome that exceeds 255.
Such big numbers don't occur usually, since Chrome just uses an internal counter that gets incremented for every new device, but nevertheless these values may be reached if the Chromebook doesn't get rebooted over a long period of time and the user plugs and unplugs devices many times in between.
The crash happens due to hitting the assertion when the value received from Chrome cannot be casted into the 8-bit number used by libusb:
https://github.com/GoogleChromeLabs/chromeos_smart_card_connector/blob/d48f773553a9441846b2e0a1371cc5feb580dfdb/third_party/libusb/naclport/src/libusb_over_chrome_usb.cc#L521
As was written in the comment in the code, one way for fixing this problem is adding code that starts reusing the device numbers once we hit the boundary (e.g., by keeping the track of the most recently used device numbers).

This would be very handy as I've run into this a few times on my Chromebooks. I move my Yubikey between several devices during the day for 2FA/GPG/SSH and after a while that leads to my Chromebook not properly interacting with the Yubikey for SSH to other machines or into Crostini if the Smart Card Connector fails. I've recently disabled Smart Card Connector temporarily due to #122 where it prevents mapping a Yubikey directly into Crostini with VMC but it would be nice to have this bug sorted as well for the times where I just need to SSH to another host and not Crostini.
I might be an edge case since I have a Chromebook with tons of RAM that I rarely reboot since more RAM == more open tabs and I have to rely on an extension or two to make closing before a reboot and reopening all of them after a reboot less onerous, but they typically handle per window tab lists and I may have a bunch of tabs "saved" previously by closing them using an extension that I wouldn't want to reopen after a reboot making the session restores more tricky.

Chrome is repeatedly crashing on my iPad Pro ,please rectify it regards nawabdera

Chrome is repeatedly crashing on my iPad Pro ,please rectify it regards nawabdera

NAWABDERA1, please use the standard bug report channel for Chrome issues: https://support.google.com/chrome/answer/95315


Enable custom cv iterator

Hi, it appears that a train/test split is used as cross validation.  I would like to see DT have a customer cv iterator for more CV folds.
System information

DeepTables version (you are using):  <IP_ADDRESS>
Are you willing to contribute it (Yes/No):  Yes

Thank you for your consideration

Hi jmrichardson ,
Here is an example of how to use cross-validation：
https://github.com/DataCanvasIO/DeepTables/blob/d929f40d092778914ab9e56d0f9d3829854ea7ac/deeptables/tests/models/deeptable_multilabel_test.py#L31-L47
Since input param X_test of dt.fit_cross_validation(X_test=df, ...)  has no y, it not used to train model.
Does it make sense?


cargo run doesn't work

hi, according to the readme cargo run is supposed to run the bin file generated by cargo bootimage using qemu.
I get an error if I run cargo bootimage
cargo bootimage
WARNING: `CARGO_MANIFEST_DIR` env variable not set
Building kernel
warning: unused manifest key: build
warning: unused manifest key: unstable
   Compiling os v0.1.0 (C:\Users\grayd\IdeaProjects\os)
error: language item required, but not found: `eh_personality`

error: could not compile `os` due to previous error
Error: Kernel build failed.
Stderr:

when i run cargo run -Zbuild-std  --target x86_64-os.json I get the expected output.
This is the error i get after running cargo run, now that i have run cargo bootimage successfully and created the .bin file.
cargo run
warning: unused manifest key: build
   Compiling os v0.1.0 (C:\Users\grayd\IdeaProjects\os)
error: language item required, but not found: `eh_personality`

error: could not compile `os` due to previous error

this is the following cargo.toml
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[build]
target = "x86_64-os.json"

[unstable]
build-std-features = ["compiler-builtins-mem"]
build-std = ["core", "compiler_builtins"]

[dependencies]
bootloader = "0.9.8"

I think i followed all the instructions. I did also have to run rustup component add llvm-tools-preview, i think to run cargo bootimage.
My code is at https://github.com/GraydenH/os if you want to try reproduce it,

This is after completing post 2 and trying to run it

The build and unstable sections need to go into .cargo/config.toml and not Cargo.toml.

@bjorn3 Hi, now i get a new error. I deleted my target folder and re ran build, bootimage and run. Build and bootimage were fine but run doesn't work.
C:\Users\grayd\IdeaProjects\os>cargo build
   Compiling compiler_builtins v0.1.52
   Compiling core v0.0.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core)
   Compiling bootloader v0.9.19
   Compiling rustc-std-workspace-core v1.99.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\rustc-std-workspace-core)
   Compiling os v0.1.0 (C:\Users\grayd\IdeaProjects\os)
    Finished dev [unoptimized + debuginfo] target(s) in 28.72s

C:\Users\grayd\IdeaProjects\os>cargo bootimage
WARNING: `CARGO_MANIFEST_DIR` env variable not set
Building kernel
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Building bootloader
   Compiling compiler_builtins v0.1.52
   Compiling core v0.0.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core)
   Compiling serde v1.0.116
   Compiling bitflags v1.2.1
   Compiling llvm-tools v0.1.1
   Compiling x86_64 v0.13.2
   Compiling toml v0.5.6
   Compiling bootloader v0.9.19 (C:\Users\grayd\.cargo\registry\src\github.com-1ecc6299db9ec823\bootloader-0.9.19)
   Compiling rustc-std-workspace-core v1.99.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\rustc-std-workspace-core)
   Compiling bit_field v0.9.0
   Compiling zero v0.1.2
   Compiling usize_conversions v0.2.0
   Compiling rlibc v1.0.0
   Compiling fixedvec v0.2.4
   Compiling xmas-elf v0.6.2
warning: use of deprecated macro `llvm_asm`: will be removed from the compiler, use asm! instead
  --> src\main.rs:45:5
   |
45 |     llvm_asm!("call $1; ${:private}.spin.${:uid}: jmp ${:private}.spin.${:uid}" ::
   |     ^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated macro `llvm_asm`: will be removed from the compiler, use asm! instead
  --> src\main.rs:91:5
   |
91 |     llvm_asm!("mov bx, 0x0
   |     ^^^^^^^^

warning: `bootloader` (bin "bootloader") generated 2 warnings
    Finished release [optimized + debuginfo] target(s) in 37.85s
Created bootimage for `os` at `C:\Users\grayd\IdeaProjects\os\target\x86_64-os\debug\bootimage-os.bin`

C:\Users\grayd\IdeaProjects\os>cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `target\x86_64-os\debug\os`
error: could not execute process `target\x86_64-os\debug\os` (never executed)

Caused by:
  %1 is not a valid Win32 application. (os error 193)

You are missing
[target.'cfg(target_os = "none")']
runner = "bootimage runner"

in .cargo/config.toml. These lines cause cargo run to run the produced kernel inside a virtual machine instead of directly as executable.

Thanks,
I get a new error after cleaning and rerunning

C:\Users\grayd\IdeaProjects\os>cargo build
   Compiling compiler_builtins v0.1.52
   Compiling core v0.0.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core)
   Compiling bootloader v0.9.19
   Compiling rustc-std-workspace-core v1.99.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\rustc-std-workspace-core)
   Compiling os v0.1.0 (C:\Users\grayd\IdeaProjects\os)
    Finished dev [unoptimized + debuginfo] target(s) in 21.85s

C:\Users\grayd\IdeaProjects\os>cargo bootimage
WARNING: `CARGO_MANIFEST_DIR` env variable not set
Building kernel
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
Building bootloader
   Compiling compiler_builtins v0.1.52
   Compiling core v0.0.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core)
   Compiling serde v1.0.116
   Compiling bitflags v1.2.1
   Compiling llvm-tools v0.1.1
   Compiling x86_64 v0.13.2
   Compiling toml v0.5.6
   Compiling bootloader v0.9.19 (C:\Users\grayd\.cargo\registry\src\github.com-1ecc6299db9ec823\bootloader-0.9.19)
   Compiling rustc-std-workspace-core v1.99.0 (C:\Users\grayd\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\rustc-std-workspace-core)
   Compiling zero v0.1.2
   Compiling bit_field v0.9.0
   Compiling fixedvec v0.2.4
   Compiling usize_conversions v0.2.0
   Compiling rlibc v1.0.0
   Compiling xmas-elf v0.6.2
warning: use of deprecated macro `llvm_asm`: will be removed from the compiler, use asm! instead
  --> src\main.rs:45:5
   |
45 |     llvm_asm!("call $1; ${:private}.spin.${:uid}: jmp ${:private}.spin.${:uid}" ::
   |     ^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated macro `llvm_asm`: will be removed from the compiler, use asm! instead
  --> src\main.rs:91:5
   |
91 |     llvm_asm!("mov bx, 0x0
   |     ^^^^^^^^

warning: `bootloader` (bin "bootloader") generated 2 warnings
    Finished release [optimized + debuginfo] target(s) in 38.83s
Created bootimage for `os` at `C:\Users\grayd\IdeaProjects\os\target\x86_64-os\debug\bootimage-os.bin`

C:\Users\grayd\IdeaProjects\os>cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `bootimage runner target\x86_64-os\debug\os`
Building bootloader
   Compiling bootloader v0.9.19 (C:\Users\grayd\.cargo\registry\src\github.com-1ecc6299db9ec823\bootloader-0.9.19)
warning: use of deprecated macro `llvm_asm`: will be removed from the compiler, use asm! instead
  --> src\main.rs:45:5
   |
45 |     llvm_asm!("call $1; ${:private}.spin.${:uid}: jmp ${:private}.spin.${:uid}" ::
   |     ^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated macro `llvm_asm`: will be removed from the compiler, use asm! instead
  --> src\main.rs:91:5
   |
91 |     llvm_asm!("mov bx, 0x0
   |     ^^^^^^^^

warning: `bootloader` (bin "bootloader") generated 2 warnings
    Finished release [optimized + debuginfo] target(s) in 2.69s
Running: `qemu-system-x86_64 -drive format=raw,file=target\x86_64-os\debug\bootimage-os.bin`
Error: Failed to execute QEMU run command `"qemu-system-x86_64" "-drive" "format=raw,file=target\\x86_64-os\\debug\\bootimage-os.bin"`: An I/O error occured: program not found
error: process didn't exit successfully: `bootimage runner target\x86_64-os\debug\os` (exit code: 1)

i see that target\\x86_64-os\\debug\\bootimage-os.bin does exist so im not sure why it complains

You need to install qemu and have it in your PATH.

It runs now but the screen is black. When i run the post-2 branch with cargo run i get  the same thing. Is something about my environment wrong or is it something else?

Which qemu version did you install? I believe there is one specific version that has a bug causing the screen to go blank for blog os.

I have 6.0.93 I think it was the latest because i just got it off the website this week. What versions have the bug? I'd like to know which version i can use that works. Maybe it should be added to the guide/readme

it worked with the 2021-02-01 release i found here https://qemu.weilnetz.de/w64/2020/

The bug is in qemu 6.0 which you used. It is fixed in 6.1: https://github.com/phil-opp/blog_os/issues/1052#issuecomment-924303421

I think this is solved and can be closed. Thanks @bjorn3!


Update Set-CsTeamsMessagingPolicy.md

Update doc to add AllowUserDeleteChat parameter

Learn Build status updates of commit d0aaecd:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

skype/skype-ps/skype/Set-CsTeamsMessagingPolicy.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

#sign-off

per Teams conversation with PM, who confirmed accuracy of changes, and a sense of urgency in publishing, I'm waiving our usual reviews to merge this PR.


Selecting date programatically while it is a placeholder selects both the placeholder and the real cell in the next/ prev month.

As you see in the screenshot. calling selectDate(date) for  days 27, 28, 29 then 30.
When called it for 27 selected both 27 and the place holder 27  in the next month. then called it for 28 , deselcted 27 but left 27 place holder selected then selected both 28 and its place holder in next month... .

Working on it, Thanks!

Hi @IsHass
The original idea of this is to not select the counterpart placeholder cell in Full-Screen mode. What do you think?

@WenchaoIOS yes that's what I want. when I call selectDate(date) this day should be selected only and not the placeholder.
Thanks.


Chocolatey: Failed to update package

2022-06-20 08:03:53.422 +02:00 [Information] Downloading presencelight 64 bit
from 'https://github.com/isaacrlevin/presencelight/releases/download/Desktop-v5.5.17/PresenceLight.5.5.17-x64.zip'
2022-06-20 08:03:56.757 +02:00 [Information]
2022-06-20 08:03:56.757 +02:00 [Information] Download of PresenceLight.5.5.17-x64.zip (94.2 MB) completed.
2022-06-20 08:04:00.420 +02:00 [Error] ERROR: Checksum for 'C:\Users\lucvd\AppData\Local\Temp\chocolatey\PresenceLight\<IP_ADDRESS>\PresenceLight.5.5.17-x64.zip' did not meet 'AC5ADF6A1032E7A8B7958D0FF6A4A7AF1C65ADDE7C97F4257FE0B089B552C949' for checksum type 'SHA256'. Consider passing the actual checksums through with --checksum --checksum64 once you validate the checksums are appropriate. A less secure option is to pass --ignore-checksums if necessary.
2022-06-20 08:04:00.586 +02:00 [Error] The upgrade of presencelight was NOT successful.
2022-06-20 08:04:00.586 +02:00 [Error] Error while running 'C:\ProgramData\chocolatey\lib\PresenceLight\tools\ChocolateyInstall.ps1'.
See log for details.
2022-06-20 08:04:04.590 +02:00 [Warning]
Chocolatey upgraded 0/1 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
2022-06-20 08:04:04.590 +02:00 [Information]
2022-06-20 08:04:04.590 +02:00 [Error] Failures
2022-06-20 08:04:04.590 +02:00 [Error]  - presencelight (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\PresenceLight\tools\ChocolateyInstall.ps1'.
See log for details.
2022-06-20 08:04:49.903 +02:00 [Warning] Failed to update "PresenceLight". Errors: ["
You have PresenceLight v<IP_ADDRESS> installed. Version <IP_ADDRESS> is available based on your source(s).", "ERROR: Checksum for 'C:\Users\lucvd\AppData\Local\Temp\chocolatey\PresenceLight\<IP_ADDRESS>\PresenceLight.5.5.17-x64.zip' did not meet 'AC5ADF6A1032E7A8B7958D0FF6A4A7AF1C65ADDE7C97F4257FE0B089B552C949' for checksum type 'SHA256'. Consider passing the actual checksums through with --checksum --checksum64 once you validate the checksums are appropriate. A less secure option is to pass --ignore-checksums if necessary.", "The upgrade of presencelight was NOT successful.", "Error while running 'C:\ProgramData\chocolatey\lib\PresenceLight\tools\ChocolateyInstall.ps1'.
See log for details.", "
Chocolatey upgraded 0/1 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).", "Failures", " - presencelight (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\PresenceLight\tools\ChocolateyInstall.ps1'.
See log for details."]

@gep13 any idea what could be going wrong here?

@isaacrlevin this error indicates that the zip file being downloaded from GitHub doesn't have the same SHA that is defined within the package.
To confirm this, I went and downloaded the zip file from here:
https://github.com/isaacrlevin/presencelight/releases/download/Desktop-v5.5.17/PresenceLight.5.5.17-x64.zip
and ran Get-FileHash against is, which produced:
Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
SHA256          E9F47CE46FCAC8494CB2E40CF1E243B8D4AA0CDABFE164BF3E4D8089C5330262       C:\users\gep13\downloads\PresenceLight.5.5.17-x64.zip

And comparing this to the SHA that is defined within the package, you can see that they don't match:
https://community.chocolatey.org/packages/presencelight#files
checksum64     = "AC5ADF6A1032E7A8B7958D0FF6A4A7AF1C65ADDE7C97F4257FE0B089B552C949"

In the interests of security, since the SHA's don't match, it halts when attempting to install the package.  You can workaround this using the --ignore-checksums option, however, the preferred way to workaround this would be to use the --checksum64 option, passing in the exact checksum of the file that is being downloaded.
More information about the above in the blog post that I wrote about this a while back:
https://www.gep13.co.uk/blog/chocolatey-error-hashes-do-not-match
Has the zip file on GitHub changed since the Chocolatey package was published?  If so, you will need to push a new package version to reflect this change.

@LucVanDyck can you install the new version from Choco and let me know if it works

@LucVanDyck can you install the new version from Choco and let me know if it works
Yes, now it works 🎉


quic: Use expected_server_preferred_address instead of sent_server_pr…

A recent QUICHE change renamed the method sent_server_preferred_address to expected_server_preferred_address, but left the old method around. This PR removes a call to the old method which will allow the QUICHE old method to be removed.
Risk Level: N/A - No behavior change
Testing: N/A
Docs Changes: N/A
Release Notes: N/A

/assign @danzh2010

/assign @yanavlasov


logging methods update

internally, npinit uses a logging process with custom settings for practically everything.
need to work this out.

will most likely use internal code to replace the display logging within npinit. for all intents and purposes, they are identical.

need to finish the log.event method


[BUG] Plugin incompatible with latest obsidian installer version

Same issue here !

Same here!

+1

Sam here. Big issue

The next update on zotero and obsidian plugin will arrive in a week or two. It's a rather major update, sorry for the long wait!

@aidenlx Hi, is there any update on the progress with this next update? I am eager to try the plugin, but due to this issue, I cannot.

same issues！！

v1.0.0 support latest obsidian version, make sure to update zotero plugin manually as well!


Alewis/generate site

Add a flag --site with special logic for which template to generate.

closing in favor of a larger PR sorrynotsorry


Rename JunctionNode -> IntersectionNode

I choose that name without thinking it through and it should be an intersection and not a junction (I foresee we will want to have proper junctions in the near future). As usual, check the comments, tests, READMEs, etc are updated accordingly.

Addressed in https://github.com/ekumenlabs/terminus/pull/109


Update GHA linux runners to Ubuntu 20

GHA deprecate Ubuntu 18 runners,
We update the CI to use Ubuntu 20 runners
See https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/
When I used the default compiler (GCC 9.5) it used lto (link time optimization) and it failed on an internal error.
So I forced it to use GCC 7 which compile.

@OhadMeir ready for review.
I will add myself a comment to add a comment for the lines that set the required GCC version


change all "Shoot Name" to "Session Name"

em changed on printed order form only

ash changed elsewhere. done.


createRevision() does not throw checked exceptions

It does not throw in Document and so it shouldn't throw in SavedRevision.

Hi @sergio91pt
Thank you very much for your PR.
We have a couple of requests before merging your PR.

Could you please sign Sign the contributor license agreement  => https://github.com/couchbase/couchbase-lite-ios/wiki/Sign-the-Contributor-License-Agreement

Thanks!

Signed it last week, its under Sérgio Faria:

@sergio91pt Thank you very much for your contribution!


Snap murals and clocks to walls

In the chat, people asked about clocks, murals, and graffitti. Probably also mosaics?

amenity=clock is often mounted on a pole though, so it should be snapped to wall only if user selected support=wall_mounted in fields
(in fact, I'd wager that any object that has support=wall_mounted field should be snapped to the wall!)

Okay, did exactly that in 5.2. Thanks @mnalis for the clarification.


Group name is not populated from template params to constants

Related with #2 by @rynowak
"You might want the plural to be a template parameter as well instead of a hardcoded s - https://github.com/Xabaril/lucecu/blob/master/src/templates/k8soperator/Content/Deployment/{{crdname}}_crd_definition.yaml#L8"
Group name is already a template parameter, but is not populated yet to constants

Pull request https://github.com/Xabaril/lucecu/pull/16

Closed by #16


Add From impl for Title

A small quality of life improvement
So that instead of doing
let layout = Layout::new().title(format!("{title}").as_str().into());

I can now do
let layout = Layout::new().title(format!("{title}").into());

This is even better! Thanks @mfreeborn :)
 let layout = Layout::new().title(format!("{title}"));

is even cleaner now :D

I'm going to go ahead and try to fix the failing examples etc. if that's alright with you

Yes, start going for it by all means! I ran out of time updating the remaining examples etc last night. I also intend to give LegendGroupTitle the same treatment, which we might as well do in this PR.

From: Filip T @.>
Sent: Tuesday, August 8, 2023 5:48:05 PM
To: igiagkiozis/plotly @.>
Cc: Michael Freeborn @.>; Mention @.>
Subject: Re: [igiagkiozis/plotly] Add From impl for Title (PR #154)
I'm going to go ahead and try to fix the failing examples etc. if that's alright with you
—
Reply to this email directly, view it on GitHubhttps://github.com/igiagkiozis/plotly/pull/154#issuecomment-1669969290, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHSVKWHCCN2XMTBK3FYCUPDXUJUULANCNFSM6AAAAAA22XQOCE.
You are receiving this because you were mentioned.Message ID: @.***>

Glad to see the development of this project continuing <3
Decided to sync the changes of this PR with the main branch and adjust some things. Let me know what you think

Seems like I missed some things... Now the clippy tests shouldn't fail

Glad to see the development of this project continuing <3
Decided to sync the changes of this PR with the main branch and adjust some things. Let me know what you think

Thanks for updating. Will have a look

@fsktom, thank you for fixing all the tests and clippy. I've just merged it.

Thank you so much! :D


chore(main): release 1.0.0

:robot: I have created a release beep boop
1.0.0 (2024-05-20)
Features

add .env.dist (2763b2b)
add authorize method to GQL Mutation (b666e36)
add context (a0a16bc)
add CORS as middleware (90acebd)
add delete event mutation (b29c5f2)
add docker compose support (63e3eb5)
add dotenv files support (84ff1d2)
add ent support (e56a816)
add gaper binary file to gitignore (cf76247)
add getting timeline events (f4a940d)
add google api client (4b32c06)
add isNew property to user type (1e02395)
add method for addint timeline events (6543138)
add ordering to list of events (6157264)
add postgres settings to config (3d1f0c9)
add README.md (238cbd2)
add relation to timeline and user (74610f7)
add sentry support (a6ead27)
add show time to entity type (9fc0c20)
add simple graphql server (7e2b7bb)
add tag entity (ab51349)
add tag repository to sl and to model (fd58290)
add tags for added event (fb4f31f)
add tags to add event GQL (e99701a)
add tags to valid GQL input (4d483f9)
add timeline and event entities (6dffb41)
add timeline list to user GQL (fe50d64)
add timeline model (3ccd5eb)
add timeline mutation (08aaa0f)
add type to timeline event entity (2b9c44d)
add url by GQL (ac960ed)
add url field to db (069d60d)
add vscode support (96a77e5)
converting timelines (0ea8e0c)
create hello world (7a66f29)
create user on request to API (78f42fe)
etract token from request (48ae7ee)
expose tags for every event entity (5dc4588)
extract google user from payload (642a048)
return user by authorization mutation (33ae550)
save description (047691f)
saving title (9705173)
upset tag (214d3bf)

Bug Fixes

add avatar to GQL User type (19d4937)
add error to every methods of interfaces (2cdc0e0)
add validation to add timeline method (eea668e)
disable loading vscode config on nvim with neoconfig plugin (16db420)
fix add timeline GQL method (e53609c)
fix adding empty tags (220d0b9)
fix adding tag (5acfbeb)
fix application (171b5ac)
fix application (4eee788)
fix authorization (350e23c)
fix authorize middleware to use on post requests only (eb0d043)
fix authorize midleware to use on post requests only (4347ba8)
fix build (5c10aa9)
fix cicd (c8e7771)
fix cicd build (27cd23b)
fix docker compose (84695c9)
fix empty tags for add event GQL (6325776)
fix expose empty tags (f0d8ae0)
fix expose tags (da66098)
fix gql generation (06df58c)
fix mod (6f8dbe6)
fix reading dotenv variables (353dd95)
fix showTime input argument (02ca3c9)
fix types of serviceLocator (9a37a11)
fix URL validation (2a51dbf)
fix vscode settings (35761c2)
make user repository implementation as private (c9c28a5)
remove id from GQL types (1255bf9)
remove useless resolvers from DI (dde5554)
sanitize text inputs (c255ba7)
update time when user was authorized (a887f6f)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wert2all/timeline-backend/releases/tag/v1.0.0 :sunflower:


does the authors have any plan to release the code about MiniGPT-v2 for Image captioning and Baichuan2 for Query debiasing?

oh, that sounds good, I am looking forward to it.

oh, that sounds good, I am looking forward to it.

done


/api/agent/agent-llm/command error 500

500Undocumented | Error: INTERNAL SERVER ERRORResponse bodyDownload{   "message": "Internal Server Error" } Response headers access-control-allow-origin: *  connection: close  content-length: 37  content-type: application/json  date: Sun,23 Apr 2023 15:20:17 GMT  server: gunicorn
Sorry great work on this it's amazing what you have done. I'm sure I need to do something differently, but I cannot get the commands to load using the docker or the local.

I am not sure that the latest version will fix the issue for sure or not, but we're not using gunicorn anymore, so it may be resolved in the latest.

The new pull worked! thank you


ci: support aarch64 builds

update github actions workflows and goreleaser to support aarch64.

Summary
Add aarch64 build and goreleaser support

Related issue :
fixes #305

hi @waveywaves thanks for this PR.. can I ask you just one small amed signin your commit? git commit --amend --signoff should be enough

hey diego, the commit was already signed off but I signed it and committed again

mmm, weird, is not picking it up in the check... never mind, I'll merge it anyway, sorry for the noise

@dciangot might be a gpg key issue on my end


Remove rmDir references

rmDir is not available on the node version used by the actions runner.
Instead, use the del package. It is safe, well-tested, and
cross-platform. Also, downgrade the @types/node package so that it more closely reflects what is being run on the server.
This should fix the failing tests on main.
Interestingly, there are several other places in the code where rmDir is being used, but it is not causing problems because (I think) these code paths are only hit by the runner.
Merge / deployment checklist

[x] Confirm this change is backwards compatible with existing workflows.
[n/a] Confirm the readme has been updated if necessary.
[n/a] Confirm the changelog has been updated if necessary.

Looks like we need a force flag to delete outside the current directory: https://github.com/github/codeql-action/runs/4464110110?check_suite_focus=true#step:7:4172

we've introduced the (very reasonable) dependency.

The dependency already existed. It's just that it was transitive. I had to make it a top level dependency in order to be allowed to import it.

Looks like we need a force flag to delete outside the current directory: https://github.com/github/codeql-action/runs/4464110110?check_suite_focus=true#step:7:4172

Curious how this check passed: https://github.com/github/codeql-action/runs/4462419683?check_suite_focus=true
which is the same job.

The dependency already existed. It's just that it was transitive.

Ah, even better :)

I think builds on main have additional data to clean up (the database that's built and uploaded for remote queries).


Thank you for your replay.
i'll try clean project.
thanks.
2018-04-08 17:45 GMT+09:00 JAYAndroid<EMAIL_ADDRESS>
grade clean assemblePlugin执行后，编译报错
ource/r/release/android/support/constraint/R.java:67: 错误: 不兼容的类型:
<空值>无法转换为int
public static final int[] ConstraintLayout_Layout = { null, null, null,
null, null, 0x7f010000, 0x7f010001, 0x7f010002, 0x7f010003, 0x7f010004,
0x7f010005, 0x7f010006, 0x7f010007, 0x7f010008, 0x7f010009, 0x7f01000a,
0x7f01000b, 0x7f01000c, 0x7f01000d, 0x7f01000e, 0x7f01000f, 0x7f010010,
0x7f010011, 0x7f010012, 0x7f010013, 0x7f010014, 0x7f010015, 0x7f010016,
0x7f010017, 0x7f010018, 0x7f010019, 0x7f01001a, 0x7f01001b, 0x7f01001c,
0x7f01001d, 0x7f01001e, 0x7f01001f, 0x7f010020, 0x7f010021, 0x7f010022,
0x7f010023, 0x7f010024, 0x7f010025, 0x7f010026, 0x7f010027, 0x7f010028,
0x7f010029, 0x7f01002a, 0x7f01002b };
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/didi/VirtualAPK/issues/200, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADBDwZzHXMtXnC-A1cJrD74CGiHSo3Nkks5tmc44gaJpZM4TLc2_
.

使用更新的版本构建。


Icon Request: Patreon

An icon to represent Patreon, useful on webpages

Seconding this! Patreon recently did a complete rebrand — their new logomark looks like this:

https://www.patreon.com/brand/guidelines

I tried to copy their new logo's dimensions while still following the rules, how does this look @colebemis ?

here's a Figma file as well:
patreon.fig.zip

Just some small changes to @csandman's icon so that it follows #171 ... what does everyone think? Figma
cc @locness3 @ahtohbi4

Maybe use 2px spacing

@locness3 Updated!

That looks not aligned with the grid.

@locness3 It is aligned with the grid ... it looks off because in order to centre it I had to adjust it to be on half-squares (if that makes sense)

Well, it should be on squares I think

@locness3 Does it matter? As long as it's centered right? I will have to adjust the circle size in order to be on squares and have a 2px distance

idk

@locness3 I'll go ahead and open a PR

Ok

@johnletey It does matter in general that the pixels are aligned correctly because on lower resolution devices, misaligned SVGs will look fuzzy.  I'm also curious what rules you believe my original version to break?

@csandman It wasn't more of rule-breaking, and more of that the icon needed to be centered and I changed it so that the line and the circle were 1px away from each other ... although it turns out that @locness3 wanted them to be 2px away.


Add gist:_year and Maybe gist:_month, gist:_hour, gist:_second (but as a gist:DurationUnit)

I needed gist:_year in a recent project, but it doesn't exist. Only gist:_day, gist:_minute, and gist:_millisecond exist.
Note: gist:_second exists but it is not a gist:DurationUnit but a gist:BaseUnit. Maybe it should be renamed to gist:_base_unit_second so that gist:_second can be used for gist:DurationUnit. Other gist:BaseUnits should be updated to following whatever naming convention is used for gist:BaseUnits.
I defined it as follows (used a different prefix in the project):
gist:_year
	a
		owl:NamedIndividual ,
		gist:DurationUnit
		;
	skos:definition "A duration unit that is 365.256 days long."^^xsd:string ;
	skos:prefLabel "year"^^xsd:string ;
	# Based on the number of ephemeris days for the J2000.0 epoch in a [sidereal
	# year](https://en.wikipedia.org/wiki/Sidereal_year) rounded to 3 decimal
	# points.
	# 365.256 * 86400 = 31,558,118.4
	gist:conversionFactor "31558118.4"^^xsd:double ;
	gist:hasBaseUnit gist:_second ;
	gist:unitSymbol "yr"^^xsd:string ;
	.

Consider adding gist:_month and gist:_hour as gist:Duration units too.

First, gist:_second is inferred to be a DurationUnit, so we are good there.
The day, minute and millisecond units were only needed for precision for TimeInstant which is not gone. So they are no longer needed in gist. They are not used any more.
There is a separate issue for whether we want to include certain units beyond the base units. If we have day, millisecond and hour, where do we stop?  Fortnight? Week?   We are planning to have some sample units of this sort in a separate file.
Month is a problem, since months vary in length.
PROPOSAL:

don't include any non-base units in gistCore
have a separate file for commonly used units across multiple enterprises, say the duration units ,as well as maybe speed, area, acceleration etc.
Have a larger file for any number of other less generic units from specific fields and industries. This file could import the other one.

ALSO:

I think this is already mentioned in another issue.

First, gist:_second is inferred to be a DurationUnit, so we are good there.

Ah, I ran the Hermit reasoner in Protege and now I see that it is inferred as a DurationUnit.
Is it because a DurationUnit has equivalent class of
gist:SimpleUnitOfMeasure and (gist:hasBaseUnit value gist:_second)

and gist:_second has rdf:type of gist:BaseUnit which is a subclass of gist:SimpleUnitOfMeasure and it also has gist:hasBaseUnit of gist:_second?

Yes, that is why. You can verify that using the explain button in Protege.

You can verify that using the explain button in Protege.

Thanks for pointing this out!

@dhurlburtusa
This is subsumed by #697


[1.4.x] Don't initialize symbols before finding classfile

This is a backport of https://github.com/sbt/zinc/pull/950
Previously, we would complete symbols before trying to find the
associated classfile. This is problematic for symbols where we don't
have the classfile (for instance, when compiling with Pants'
strict_deps enabled).
This initialization was introduced in #758, to workaround a scalac bug
where the associated classfile of a symbol wouldn't be set before the
symbol is completed. This bug has been fixed in Scala 2.12.12 and later
(see scala/scalac#8889).
Fixes #949

To validate this pull request against other sbt modules, please visit this link.


[RFC] UMA Universal Modular Accelerator Interface

opening PR for UMA pre-RFC

cc @comaniac @csullivan @jroesch @tmoreau89 @masahi

One thing I'd like anwsered here what sort of control it will allow on the passes run there : MergeComposite, AnnotateTarget, MergeCompilerRegions and ParititionGraph.

So far we had planned to standardize on MergeComposite, AnnotateTarget, MergeCompilerRegions and ParititionGraph. To get a better overview I extracted the partitioning flows of existing BYOC targets:

BYOC Backend
Pre Partition Passes
Partition
Post Partition Passes

arm_compute_lib
InferType,
MergeComposite, AnnotateTarget, PartitionGraph

bnns
InferType, FoldConstant, FoldScaleAxis, DynamicToStatic, AlterOpLayout, FoldConstant,
MergeComposite, AnnotateTarget, PartitionGraph

cmsisnn

MergeComposite, AnnotateTarget, PartitionGraph,
GenerateCMSISNNConstants, ScalarToTensorConstants, ExtractConstantsFromPartitionedFunction

cutlass
SimplifyInference, FoldConstant, FoldScaleAxis,
MergeComposite, AnnotateTarget, PartitionGraph

dnnl

MergeComposite, AnnotateTarget, MergeCompilerRegions, PartitionGraph

ethosu

MergeComposite, AnnotateTarget, MergeCompilerRegions, PartitionGraph,
preprocess_ext_io

tensorrt
RemoveDropoutPass, RemoveUnusedFunctions, ConvertLayout, FoldConstant,
AnnotateTarget, MergeCompilerRegions, PartitionGraph

vitis_ai
RemoveUnusedFunctions, ConvertLayout, FoldConstant, InferType,
("VitisAIAnnotationPass"), MergeCompilerRegions, PartitionGraph,
RemoveUnusedFunctions, ConvertLayout, FoldConstant

Looking at the existing backends it might make sense to make MergeCompilerRegions optional. We probably do not want to support custom compiler annotations as used in vitis_ai target.

@cgerum thanks for detailed analysis!
Im wondering whether should we provide an optional partitioning hook as well -- so then it can be anything and let the default be a Sequential of MergeComposite, AnnotateTarget, MergeCompilerRegions, ParititionGraph. WDYT ?

Im wondering whether should we provide an optional partitioning hook as well -- so then it can be anything (i.e. any Sequential) and let the default be a Sequential of MergeComposite, AnnotateTarget, MergeCompilerRegions, ParititionGraph. WDYT ?

Considering how partitioning is handled in #62 I would probably prefer a more declarative way of specifying different partitioning patterns. @MichaelJKlaiber @PaulPalomeroBernardo

One overall question I have is whether this proposal is strictly limited to accelerators or whether it could also be used by any back-end that leverages the target hook functionality? For example, it seems possible to register kernel libraries (e.g. CMSIS-NN) using a similar interface?

@lhutton1, I agree in general. The primary focus of UMA at the moment is accelerators. It might make sense to bear library integration in mind. In general, I see that it should be possible. The main difference might be the choice of configuration parameters needed.
@cgerum @PaulPalomeroBernardo  what are your thoughts here?
@lhutton1, if you have suggestion or concrete examples, feel free to share them

Michael

Thanks @MichaelJKlaiber, that makes sense. So I was wondering if this is the case, perhaps in the future this interface is used by other backend's (not accelerators) we would need to think about renaming UMA to something more generic e.g. UMB Universal Modular Backend - I'm not the best with names. I'm wondering if this could easily be done in the future?

Thanks @MichaelJKlaiber, that makes sense. So I was wondering if this is the case, perhaps in the future this interface is used by other backend's (not accelerators) we would need to think about renaming UMA to something more generic e.g. UMB Universal Modular Backend - I'm not the best with names. I'm wondering if this could easily be done in the future?

@lhutton1, I think for naming there are no limits for creativity, e.g. it could be UMA: Universal Modular bAckend 😄

@MichaelJKlaiber is this one ready for another look?

@MichaelJKlaiber, I did another pass. It is looking good!
From my side, Ideally, there are only two things that we need to agree upon on the design (which relates two outstanding conversations above).
A1. The rationale to use the int-based phase in registering passes.
A2. The methodology to attach target-specific options that affects multiple passes in the lowering.
For A2, it would be bit important to reach a conclusion where we could point future contributors/users who want to use UMA for upstream/downstream development, respectively.

for A2, we should consider the type of options present:

options that serve to specify how a pass runs should eventually find their way into the active PassContext
options which are constraints can be attached to Target or similar.

I think we could use Target-registered compilation flow customization to hold these options. What do you think?

@MichaelJKlaiber @cgerum any more thoughts on @manupa-arm 's two questions above? would be great to get this RFC in!

@areusch and @MichaelJKlaiber, I agree with using Target-registered compilation flow customization.
I am struggling to how to connect that with :
TVM_REGISTER_GLOBAL("relay.backend.contrib.uma.RegisterTarget")
    .set_body_typed([](String target_name){
        ::tvm::TargetKindRegEntry::RegisterOrGet(target_name)
        .set_name()
        .set_device_type(kDLCPU)
        .add_attr_option<Array<String>>("keys")
        .add_attr_option<String>("tag")
        .add_attr_option<String>("device")
        .add_attr_option<String>("model")
        .add_attr_option<Array<String>>("libs")
        .add_attr_option<Target>("host")
        .add_attr_option<Integer>("from_device")
        .set_attr<FTVMRelayToTIR>("RelayToTIR", relay::contrib::uma::RelayToTIR(target_name))
        .set_attr<FTVMTIRToRuntime>("TIRToRuntime", relay::contrib::uma::TIRToRuntime);
    });

What would be the API to add an option ?
e.g. from here
class UltraTrailBackend(UMABackend):
    def __init__(self):
        super(UltraTrailBackend, self).__init__()

        # Relay to Relay function registration
        self._register_pattern("conv1d_relu", conv1d_relu_pattern())

        self._register_relay_pass(1, ConfigGenerator())
        self._register_relay_pass(2, BufferScopeAnnotator())

        # Relay to TIR function registration
        self._register_operator_strategy("nn.conv1d", custom_conv1d_strategy)

        self._register_tir_pass(0, CodegenGenerateConfig())
        self._register_tir_pass(0, CodegenGenerateExternCalls())

        # TIR to runtime function registration
        self._register_codegen(format="c", includes=gen_includes, replace_call_extern=None)

    @property
    def target_name(self):
        return "ultra_trail"

In order to help to progress with A2,
I can think of two solutions :
A2.1 :  We could register a sub_target string (similiar to a mtriple in LLVM) and a decoder (e.g. : _register_sub_target_decoder(ultra_trail_target_decoder(str)) --> AttrDict/Dict. We could register the attr_dict as uma_attrs to the templated UMA target registration.
A2.2 : We could directly register an attr_dict. i.e. _register_target_attr(AttrDict/Dict)
I have a slight preference for A2.1 as it will only expose a string to the user in the actual compilation run.
PS :
@Mousius highlights the motivation behind making the target attributes discoverable in this new recent RFC : https://github.com/apache/tvm-rfcs/blob/1901b7361891fc7235b3bb8e80c89130cfe2c91d/rfcs/0070-target-preprocessing.md

For A1,
My personal preference is that we go for Enum based approach rather than int s.
Unless, of course, we have a good reason not to do that -- which I think should be outlined in the RFC for future reference.

for A1, what do people think of an ordering spec e.g.
self._register_relay_pass(ConfigGenerator(), before="FuseOps", after=("MergeComposite", "abcOtherPass"))

my concern is that whether int or enum, people are really expressing a dependency graph here and while we hope it is not terribly complicated, it's hard to intuit the meaning from an enum/int.
for A2, i agree with @manupa-arm 's question, but i think it would be awesome to see the prototype and we could discuss from there.

The main downside I see with this, is that all attributes are treated as strings since the type is hardcoded. However, I'm not sure if we can avoid this at all.

This could probably be solved by adding type and/or default arguments to the argument parser, e.g.:
  self._register_target_attr("ultra_trail_attr_1", default=False)

For v1 I would prefer to only support default values, and restrict supported dtypes to string, int and bool.

For A1: I would like to keep the phases. They definitely need proper documentation, but I think a handfull of phases (e.g., PRE_PARTITIONING, POST_PARTITIONING, ...) provide more orientation for new users than having to explicitly define the dependencies to other passes. We could think of also supporting the before and after options to provide more flexibility for experienced users.

I agree with @PaulPalomeroBernardo on the user perspective.
Implementing before and after options, also goes beyond the current Scope of UMA in my opinion. If we were to expose it in the UMA-API, pass dependencies should probably be implemented in TVM core.

Thanks @cgerum and @PaulPalomeroBernardo . I agree, this totally makes sense like this.
@manupa-arm @areusch is this sufficiently detailed for you? I propose to discuss outstanding topics in meeting to settle for UMA-v1. We could use the Community Meeting on May 25th. Or if these discussions are too specific for a broader audience, then we can setup a separate meeting.
What are your thoughts?

They can be used during target creation similar to other sub_target strings

ut_target = tvm.target.Target("ultra_trail -ultra_trail_attr_1=attr1 -ultra_trail_attr_2=attr2")

This could probably be solved by adding type and/or default arguments to the argument parser, e.g.:

  self._register_target_attr("ultra_trail_attr_1", default=False)

This aligns with A2.2 -- directly registering each attribute. I think this is fine for UMA-v1 and aligns with state of TVM targets today. Should we just put a note that for future considerations, to include a registration for string preprocessor (A2.1) to extract attributes ?
For A1:
I would like to keep the phases. They definitely need proper documentation, but I think a handfull of phases (e.g., PRE_PARTITIONING, POST_PARTITIONING, ...) provide more orientation for new users than having to explicitly define the dependencies to other passes. We could think of also supporting the before and after options to provide more flexibility for experienced users.

Again, I think phase approach is fine for v1 as we already have that in the core compiler (which is also int based) but I'd appreciate if we can put a "name" to ease the reasoning in future. Similarly, we could also note as future work to define dependencies on passes -- if and when the TVM core compiler improve its pass infrastructure we could be able to use that information.

This aligns with A2.2 -- directly registering each attribute.

@manupa-arm Then just for clarification a few questions because I might have misunderstood your initial idea. For A2.1 you were thinking about registering an attribute preprocessor to the target Target().add_attrs_preprocessor(Preprocessor) that would operate on a predefined attribute (e.g., -uma_attrs=<string>) by processing the <string> and creating a Dict/Map from it?
So a user would only write tvm.target.Target("ultra_trail -uma_attrs=<my custom attr string>") and in code you would access the target via target.attrs["uma_attrs"]["attr1"], target.attrs["uma_attrs"]["attr2"], ect.?

So a user would only write tvm.target.Target("ultra_trail -uma_attrs=") and in code you would access the target via target.attrs["uma_attrs"]["attr1"], target.attrs["uma_attrs"]["attr2"], ect.?

More or less yes -- maybe we could (re) use "mattr" instead of "uma_attrs" looking at other target kinds -- but in principle that is what I meant.

ok for A1 i'm good with named phases and we can modify as necessary. i think the A2.2 solution of directly registering target attrs makes sense to me. is that the direction we're aligned on here?
we can discuss this next week at the community meeting, or if we're in alignment on these two items, i think all that remains is to update the RFC to reflect the discussion here and we can approve/merge.

Apologies for not following the conversation in detail in real time. Here are some thoughts on how we can make sure an UMA-integrated accelerator is also a Collage-supported 'backend'.

The registration of patterns will need to support the existing triple of (pattern name, pattern, predicate) since the predicates are necessary to control support based on dtypes, shapes, backend version, etc. No big deal.
I'm assuming those triples will continue to end up in either the global pattern table registry, or can be otherwise retrieved by a system like Collage which wishes to bypass the 'eager' UMA partitioning with it's own search. But again no big deal, just need to know where to look.
Though not significant to Collage, I assume the order of application of the partitioning patterns matches the registration order?
Collage requires external codegen compiler names to be 1:1 with already registered target kinds with the same kind name. It also requires instances of those targets to be provided in the build targets list, even if those instances are nothing other than Target("my_backend") with no extra attributes. But the target kinds may also support additional attributes, and the various transitions into external codegen code have been changed to ensure the matching Target instance has been pushed as the Target.current() so that codegen can retrieve and extract any attributes to guide compilation. I think that matches some of the conversation above, except that the attributes can be fetched by Target.current().get_attr("foo"), but I might have missed the point in that sub-thread.
Collage assumes a regular build of an IRModule will respect any existing "Compiler" attributed functions already in the module. I think all that means is that the UMA partitioner should respect existing partitions, but otherwise trigger the appropriate custom downstream compilation, and given the partitioner uses the existing passes I think that should all Just Work.
Collage assumes it can do it's partitioning before any other backend-specific passes. I'm assuming however that some of the Relay pass phases mentioned can be before partitioning. If so I'm guessing we'd need to first apply those pre-partitioning phases in deterministic order in the hope that they sensibly compose, then partition using Collage, then run the post-partitioning phases as usual.
Collage uses the list of available Targets to guide it's search, but if I understand correctly UMA uses the registration of backends to enforce a fixed partitioning order. Perhaps this suggests the Collage partitioner should be integrated as a user-controlled alternative to the default  'eager' partitoner supplied by UMA (presumably as a loop of the usual Relay MergeComposite/AnnotateTarget/MergeCompilerRegions?/PartitionGraph passes for each backend). That way the user can use the same construct-and-register-backends-of-interest API.
I'm surprised by the emphasis on going via TIR. Are we explicitly saying any BYOC integrations which don't need/want to go via TIR don't fall under the UMA integration API? If so that will make Collage/UMA integration harder since Collage would have to account for both UMA-style and original-style integrations.

Thanks,
-m

One more collage/uma overlap aspect: Collage distinguishes 'registered' backends (ie just TargetKinds) from 'activated' backends (ie Target objects in the provided build targets). I think though the proposal here is the act of registration is also activation? I need help understanding how this will look from the user's pov in combination with targets.

Thanks @mbs-octoml for this detailed explanation. Being a Collage-supported backend is definitely something we want to achieve for UMA-integrated backends.

The registration of patterns will need to support the existing triple of (pattern name, pattern, predicate) since the predicates are necessary to control support based on dtypes, shapes, backend version, etc. No big deal.

We will add this to the pattern registration.

I'm assuming those triples will continue to end up in either the global pattern table registry, or can be otherwise retrieved by a system like Collage which wishes to bypass the 'eager' UMA partitioning with it's own search. But again no big deal, just need to know where to look.

They are registered in the global pattern table registry during backend registration but can also be accessed directly over the backend object if necessary.

Though not significant to Collage, I assume the order of application of the partitioning patterns matches the registration order?

Correct.

Collage requires external codegen compiler names to be 1:1 with already registered target kinds with the same kind name. It also requires instances of those targets to be provided in the build targets list, even if those instances are nothing other than Target("my_backend") with no extra attributes. But the target kinds may also support additional attributes, and the various transitions into external codegen code have been changed to ensure the matching Target instance has been pushed as the Target.current() so that codegen can retrieve and extract any attributes to guide compilation. I think that matches some of the conversation above, except that the attributes can be fetched by Target.current().get_attr("foo"), but I might have missed the point in that sub-thread.

I think, this works well. After the backend registration (e.g., UMABackend.register()) the target kind, which matches the required codegen compiler name, is available. From there, a target can be created (with or without attributes) and passed to the build target list.

Collage assumes a regular build of an IRModule will respect any existing "Compiler" attributed functions already in the module. I think all that means is that the UMA partitioner should respect existing partitions, but otherwise trigger the appropriate custom downstream compilation, and given the partitioner uses the existing passes I think that should all Just Work.

I agree.

Collage assumes it can do it's partitioning before any other backend-specific passes. I'm assuming however that some of the Relay pass phases mentioned can be before partitioning. If so I'm guessing we'd need to first apply those pre-partitioning phases in deterministic order in the hope that they sensibly compose, then partition using Collage, then run the post-partitioning phases as usual.

Yes, we were planning to include a pre-partitioning pass phase. Passes within one pass phase should always be executed in order of their registration.

Collage uses the list of available Targets to guide it's search, but if I understand correctly UMA uses the registration of backends to enforce a fixed partitioning order. Perhaps this suggests the Collage partitioner should be integrated as a user-controlled alternative to the default 'eager' partitoner supplied by UMA (presumably as a loop of the usual Relay MergeComposite/AnnotateTarget/MergeCompilerRegions?/PartitionGraph passes for each backend). That way the user can use the same construct-and-register-backends-of-interest API.

Currently a user needs to explicitly call partition() on the registered backend to perform the usual MergeComposite/AnnotateTarget/MergeCompilerRegions?/PartitionGraph passes plus the relevant relay pass phases (e.g., pre-partitioning).
backendA= MyUMABackendA()
backendB= MyUMABackendB()

backendA.register()
backendB.register()
mod = backendA.partition(mod)
mod = backendB.partition(mod)

As you described this would eagerly partition the graph depending on the call order of .partition(). This would actually give the user the opportunity to skip this partitioning and directly go for the Collage approach. I am not sure if this is the best solution though.

I'm surprised by the emphasis on going via TIR. Are we explicitly saying any BYOC integrations which don't need/want to go via TIR don't fall under the UMA integration API? If so that will make Collage/UMA integration harder since Collage would have to account for both UMA-style and original-style integrations.

As it is now, they would not fall under the UMA integration API. With UMA we wanted to wrap one specific BYOC integration into an easy-to-use interface and we decided to go with the target hooks via TIR (relay_to_tir, tir_to_runtime). However, if there is enough motivation we could think about adding relay_to_runtime as a second path. This would require greater changes to the current architecture so I don't see it as part of UMA v1 but we can take this into account for future development.

One more collage/uma overlap aspect: Collage distinguishes 'registered' backends (ie just TargetKinds) from 'activated' backends (ie Target objects in the provided build targets). I think though the proposal here is the act of registration is also activation? I need help understanding how this will look from the user's pov in combination with targets.

There are three steps required to make use of UMA as a user.

Create and instantiate a UMA backend backend = MyUMABackend()
Register the backend backend.register()
Apply the standard partitioning (might not be necessary with Collage)

backend.register() is registering the target kind, a pattern table, and global functions required by the UMA lowering. I think this is more or less equivalent with the Collage 'registration'. Only when the partitioning annotates a subgraph for the backend, it is 'activated'.

ok for A1 i'm good with named phases and we can modify as necessary. i think the A2.2 solution of directly registering target attrs makes sense to me. is that the direction we're aligned on here?
we can discuss this next week at the community meeting, or if we're in alignment on these two items, i think all that remains is to update the RFC to reflect the discussion here and we can approve/merge.

@manupa-arm @areusch I think, we are aligned on this. We decided to go with the enum-based approach for A1 and use A2.2 for UMA v1. I updated the RFC accordingly (Pass Phases, Target Hooks).

thanks @MichaelJKlaiber @PaulPalomeroBernardo @cgerum and others! the RFC is now merged. Please open a tracking issue and link it from this thread for discoverability.


ZendTest\I18n\Validator\FloatTest::testValidationFailures fails with icu 54

Since the update from icu 53 to 54, this test fails.
There was 1 failure:
1) ZendTest\I18n\Validator\FloatTest::testValidationFailures with data set #14 ('2 000', false, 'fr-CH')
Failed expecting 2 000 being false (locale:fr-CH)
Failed asserting that true matches expected false.

Detected in Fedora QA, see
http://koschei.cloud.fedoraproject.org/package/php-ZendFramework2

Yep. Looks like CLDR 26 (part of ICU 54) made that change. I just so happened to pick a wrong example for a validation failure! I'll put in a PR this week to remove it. Functionality still works, just a test break.


Nominatim v3.2

Docker setup with Nominatim 3.2 and Postgres 10.
This has been tested and it is currently used in a production environment

Could you please update this to 3.3?

@Zverik @dlucia  I'll apply it tonight.

Cool, thanks @winsento. I can do it as well in case you have not time for it :)

@dlucia added you to collaborators.

@all-contributors please add @dlucia for infrastructure and code.


Test noir_wasm on PR

Problem
Avoid breaking changes in noir_wasm when updating noir.
Proposed solution
Use noir-wasm-testing on PR
Alternatives considered
No response
Additional context
No response
Submission Checklist

[ ] Once I hit submit, I will assign this issue to the Project Board with the appropriate tags.

Building and testing of noir_wasm need to happen in noir-lang/noir flake.nix where wasm output needs to get defined and "check wasm" output is used for testing.

Currently working on that, on this PR https://github.com/noir-lang/noir/pull/1208
(work in progress)

Moved to this PR: https://github.com/noir-lang/noir/pull/1660