Bug: Mapping builder fails to look at include mappings when resolving class mappings

🐛 Bug Report
When resolving class mappings during the building the metamodel mapping, the builder fails to look in the include mappings for the target class mappings.
Step to Reproduce
Copy test grammar below into text mode. The grammar will compile with legend engine but will fail against studio's building of the metamodel. This is due to the builder not resolving the class mapping model_Person in the included mappings. We should look to ensure the mapping editor handles this gracefully as we don't search in the included mappings for class mappings.
Class model::SPerson
{
  firstName: String[1];
  lastName: String[1];
}

Class model::Person
{
  fullName: String[1];
}

Class model::Firm
{
  name: String[1];
  employees: model::Person[1];
}

Class model::SFirm
{
  employees: model::SPerson[1];
  name: String[1];
}

###Mapping
Mapping model::IncludeMapping
(
  *model::Person: Pure
  {
    ~src model::SPerson
    fullName: $src.firstName + ' ' + $src.lastName
  }
)

Mapping model::M2MMapping
(
  include model::IncludeMapping

  *model::Firm: Pure
  {
    ~src model::SFirm
    name: $src.name,
    employees[model_Person]: $src.employees
  }
)

Expected behavior
When building a mapping and resolving a class mapping, the builder should also look at the include mappings.

@MauricioUyaguari I think this is related to the bug where query builder not detecting certain properties as mapped, so let's prioritize this.


Add main town name to search dropdown - e.g. Nottingham (all)  before all the various Nottingham sub-divisions)

There are currently seven Nottingham Census Places on the search screen.  This is a level of detail which may not be known by the researcher - who only knows that the person they are looking for has had a life event in "Nottingham".
The "Nottingham (all)" selection would allow them to find someone with specific details / unusual name.

@geoffj-FUG will investigate

Pat
The current search is based on the PARMS.
There are 4 levels to the PARMS:
County
Registration District
Piece or sub-District
Civil Parish
Currently we use County and civil parish as our search parameters. It is possible to add District to these levels of the search and use it as a filter to narrow down the county search area. However the PARMS was never intended to be designed to do the search that you are proposing. There are however three options:
One is to use the existing search around the selected place tick under the search boxes.
The second is if we can allow the selection of more than one place name using the control key as we can in FreeBMD.
The third is to include the Registration District which will allow a broader search. This is the subject of another story. It will need to be addressed some time as it is the link to FreeBMD if we end up having a one search covers all approach.
Your problem will be addressed by the POB search which is after all the bread and butter search available on the commercial websites. POB searching is the main focus at the moment because it is so important to the future of FreeCEN.
Geoff

Pat to look at Help.


Add case projection_perspective

Hi there, can you please add some details on what this PR is about? Does it add new examples? If so, what examples?

Sorry, this is test only.  I am not upload thsi for review.


added support Neatsvor X600 vacuum

reference to #1079

i think that this integrations os not 100%
neatsvor have more than 3 vacuum power for example

@make-all please, check this device, is still not fully working, these are the debug:
2024-06-22 00:36:50.352 DEBUG (MainThread) [custom_components.tuya_local.device] Robot vacuum received {"1": true, "4": "chargego", "5": "charge_done", "6": 25, "7": 27, "8": 100, "9": "max", "10": "closed", "17": 104, "19": 204, "21": 54, "23": 141, "26": 10, "28": 0, "full_poll": true}
2024-06-22 00:36:50.353 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Considering mapping for range of value
2024-06-22 00:36:50.353 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Considering mapping for step of value
2024-06-22 00:36:50.354 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Step for value is 1 with scale 0.1
2024-06-22 00:36:50.354 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 26 value from 10 to 100.0
2024-06-22 00:36:50.354 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 25 value from None to False
2024-06-22 00:36:50.354 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] command values: ['smart', 'return_to_base', 'zone', 'clean_spot', 'partial', 'room']
2024-06-22 00:36:50.354 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] fan_speed values: ['Off', 'Low', 'Medium', 'High']
2024-06-22 00:36:50.355 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 5 value from charge_done to charged
2024-06-22 00:36:50.355 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] command values: ['smart', 'return_to_base', 'zone', 'clean_spot', 'partial', 'room']
2024-06-22 00:36:50.355 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 1 value from True to True
2024-06-22 00:36:50.355 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 4 value from chargego to return_to_base
2024-06-22 00:36:50.356 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] command values: ['smart', 'return_to_base', 'zone', 'clean_spot', 'partial', 'room']
2024-06-22 00:36:50.357 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] option values: ['Off', 'Low', 'Medium', 'High']
2024-06-22 00:36:50.357 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 10 value from closed to Off
2024-06-22 00:36:50.357 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] option values: ['Türkçe', 'English', 'Русский', 'Español']
2024-06-22 00:36:50.357 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] Robot vacuum: Mapped dps 36 value from None to unknown
2024-06-22 00:36:50.358 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 6 (sensor), unable to determine valid values
2024-06-22 00:36:50.358 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 7 (sensor), unable to determine valid values
2024-06-22 00:36:50.358 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 8 (sensor), unable to determine valid values
2024-06-22 00:36:50.358 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 17 (sensor), unable to determine valid values
2024-06-22 00:36:50.358 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 19 (sensor), unable to determine valid values
2024-06-22 00:36:50.358 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 21 (sensor), unable to determine valid values
2024-06-22 00:36:50.359 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 21 (sensor), unable to determine valid values
2024-06-22 00:36:50.359 DEBUG (MainThread) [custom_components.tuya_local.helpers.device_config] No mapping for dpid 23 (sensor), unable to determine valid values

There are no errors in that log.

sorry, got that my unit is a Neatsvor x600pro not just a x600,
would be possible to add this version on the home assistant integration?
i'm glad to help if you need anything


⚠ Thumbnails API Endpoint has degraded performance

In 78d1222, Thumbnails API Endpoint (https://thumbnails.roblox.com/v1/assets?assetIds=82738847&format=Png&isCircular=false&size=30x30) experienced degraded performance:

HTTP code: 200
Response time: 1107 ms

Resolved: Thumbnails API Endpoint performance has improved in 2eae360 after 4 minutes.


racer returned a non-zero exit code: 101 16: 0x778859cd - BaseThreadInitThunk

Received this error when using Racer with edubkendo's Atom editor plugin on Windows 7. However, "racer complete std::" in cmd works fine.

I'm not sure if this is an issue with racer or the plugin, so I've opened an issue there as well.

I have lots of these errors when typing doc comments in Atom. The notification area in it is literally overflown by error messages from the plugin.

Hi @netvl
Ah, it sounds like the plugin is calling into racer on every keypress, even when typing doc comments. I hadn't anticipated that.
It sounds like racer should handle this because I bet other plugins do this too. I'll have a think about how best to handle this situation. Thanks for the issue report

Hi @netvl
I tried wrapping the completion function in a thread in order to catch panics. Could you try again and see if this helps please?

@phildawes, sure, I'll try it when I get to my home computer.

This fixed the problem on my setup!

Indeed, I haven't seen this flood of messages for some time with the latest maser, looks like it was fixed.


A new training service implementation for Kubernetes by Petuum

Hi NNI Developers,
We are Petuum developers and we'd like to contribute a new training service to NNI 😄,  which is AdlTrainingService that implements the original KubernetesTrainingService abstraction.
Background
Recently, collaborating with a CMU lab, we released an open-source project: AdaptDL, aiming to make distributed deep learning easy and efficient in dynamic-resource environments such as shared clusters and the cloud.
It consists of a Kubernetes job scheduler and an adaptive training library. The scheduler part directly schedules the trial jobs and optimizes cluster-wide training performance and resource utilization. The added training service AdlTrainingService uses the Kubernetes client library to interact with the AdaptDL scheduler CRD, to control the lifecycles of the trials of experiments defined upon this new training service.
We have already adopted this integration on a fork for our machine-learning engineers internally, and it's very cool and useful! Therefore, we'd like to contribute back to NNI, so that we could grow our open-source community together.
Outline
While developing, we are following a principle that all changes added for the new training service does NOT affect any features that NNI already has, so that all NNI functionalities remain the same as the master.
This pull request consists of the following parts:

AdlTrainingService and its relevant features all compatible with nnictl:

Core implementation and its corresponding config schema update (We also include an example config for mnist-pytorch example.)
Internal storage (PVC) and external storage (NFS) support
Tensorboard support
Log streaming and collection support

A new in-memory data field message in job info to display the status-related short message for a trial job together with status. For example, when an AdaptDL trial job is pending status, the message can distinguish it from "waiting in the queue" or "pulling the image" etc. So NNI status and message together could be informative enough, and that users don't need to query more info directly via Kubernetes anymore.

Test
Note that, the core test cases for the new Kubernetes training service in this PR, are integration testing, that needs a Kubernetes cluster for it. If skipping integration test like the NNI Kubeflow training service, all original unit-test cases behave the same as those in master.

Hope that we could successfully merge it later after some review discussions and changes.
If you guys have any questions, let us know!
Thank you! 👍

Hello, thanks for your contribution!
We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.
For now I have following concerns about the PR:

What's the purpose of "trial job message"? Can you give an example of its content?
I don't prefer to sort sequence number of distributed intermediate results with explicit accumulator in trial SDK. The logic looks too specialized to me. I don't know how distributed trials work on other training service at this point, but I think it should either be a common feature or be handled silently in training service (nni.platform is also considered training service here).
Do you have an integration test pipeline? If it involves authentication we can discuss it the meeting.

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

What's the purpose of "trial job message"? Can you give an example of its content?

It is a short description of the status. Likewise, in Kubernetes, a pod/container status object can have a corresponding short description to explain that status. For example, a "Pending" status may have different messages (like the example mentioned in the above description). All "message" value-writes happen in adlJobInfoCollector.

I don't prefer to sort sequence number of distributed intermediate results with explicit accumulator in trial SDK. The logic looks too specialized to me. I don't know how distributed trials work on other training service at this point, but I think it should either be a common feature or be handled silently in training service (nni.platform is also considered training service here).

We are not sorting any sequence number. The additional optional helper in trial SDK is a way to guarantee the continuity of the sequence number during the (distributed) preemptive scheduling [which can be treated as a general case]:
For example, after training a model from step 0 to 6, our software may interrupt the pod to reallocate resources. However, after the pod is restarted with the newly allocated resources, the accumulator there knows we are in step 6 while the NNI considers it from 0 again. So in order to report the intermediate result with the correct sequence id, we need to get the correct value of it from the accum and reset the _intermediate_seq. In short, it requires an interface to set the value of _intermediate_seq.

Do you have an integration test pipeline? If it involves authentication we can discuss it at the meeting.

Yes. The links in the Test section in the above description) is the test case. BTW, we notice the current NNI Kubeflow test is skipped without authenticating to a Kubernetes cluster, so we mimic that treatment in our integration test case as well. On the other hand, if you guys have a Kubernetes in the Azure pipeline for testing, it'd be great you could assist us to point to it with proper authentication.

Thank you!

Ooops, seems I missed the description.
For the integration test, I suppose the test case only requires a raw kubernetes cluster without additional setup steps? If that's the case I think we can deploy the pipeline on NNI's cluster.
Please provide a pipeline script like this one, and I will try it on dev-adl branch.
For intermediate sequence, how should user create or get the accumulator object? Or if it's simply a integral number? I don't see it in the example. If it's a simple number maybe we can set it via environment variable, and only modify line 99 in trial.py.

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍
Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Please add a AdlMode.md doc under https://github.com/microsoft/nni/tree/master/docs/en_US/TrainingService, give a description for the training service and explain how to install and run.

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Thanks @scarlett2018 ! Sounds good to me

Email<EMAIL_ADDRESS>Some availabilities in Pacific Time: this Wednesday 12:00 - 2:00 PM or Thursday 10:30 - 2:00 PM or Friday 10:30 - 2:00 PM.

Please add a AdlMode.md doc under https://github.com/microsoft/nni/tree/master/docs/en_US/TrainingService, give a description for the training service and explain how to prepare environment and start experiments.

Yup, we are working on it.

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Thanks @scarlett2018 ! Sounds good to me

Email<EMAIL_ADDRESS>Some availabilities in Pacific Time: this Wednesday 12:00 - 2:00 PM or Thursday 10:30 - 2:00 PM or Friday 10:30 - 2:00 PM.

Hi @scarlett2018 , just a follow-up, would there still be a meeting sometime this week?

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Thanks @scarlett2018 ! Sounds good to me

Email<EMAIL_ADDRESS>Some availabilities in Pacific Time: this Wednesday 12:00 - 2:00 PM or Thursday 10:30 - 2:00 PM or Friday 10:30 - 2:00 PM.

Hi @scarlett2018 , just a follow-up, would there still be a meeting sometime this week?

Hi @pw2393, sorry for the delayed responds, this week actually not working well for all. I started a doodle vote for next week, and there are currently 4 options, hope some of them works well for you:
https://doodle.com/poll/5ky9scc4xmb3mkh7

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Thanks @scarlett2018 ! Sounds good to me

Email<EMAIL_ADDRESS>Some availabilities in Pacific Time: this Wednesday 12:00 - 2:00 PM or Thursday 10:30 - 2:00 PM or Friday 10:30 - 2:00 PM.

Hi @scarlett2018 , just a follow-up, would there still be a meeting sometime this week?

Hi @pw2393, sorry for the delayed responds, this week actually not working well for all the reviewers. I started a doodle vote for next week, and there are currently 4 options, hope some of them works well for you:
https://doodle.com/poll/5ky9scc4xmb3mkh7

Thanks @scarlett2018.  selected.

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Thanks @scarlett2018 ! Sounds good to me

Email<EMAIL_ADDRESS>Some availabilities in Pacific Time: this Wednesday 12:00 - 2:00 PM or Thursday 10:30 - 2:00 PM or Friday 10:30 - 2:00 PM.

Hi @scarlett2018 , just a follow-up, would there still be a meeting sometime this week?

Hi @pw2393, sorry for the delayed responds, this week actually not working well for all the reviewers. I started a doodle vote for next week, and there are currently 4 options, hope some of them works well for you:
https://doodle.com/poll/5ky9scc4xmb3mkh7

Thanks @scarlett2018. Slot is selected. Could you send us the meeting invite via email?

Thanks @pw2393, this Wed does not work out in the end, I pick up a same time slot for next Wed and send out the invitation, hope it work for you.

We want to have a review meeting with you to discuss the integration. Our PM will contact you soon.

Cool! Thanks. Looking forward to it. 👍

Hi @pw2393 - This is Scarlett, PM of the project. Firstly, thanks for contributing to NNI, glad to see another training service that can benefit a new AI platform user. As this PR is pretty big, shall we book a 30 mins to 1 hour PR review sometime this week? I will invite NNI dev who works on Training Service part as well. Let me know your email and available timeslot(s) for sending out the calendar invitation.

Thanks @scarlett2018 ! Sounds good to me

Email<EMAIL_ADDRESS>Some availabilities in Pacific Time: this Wednesday 12:00 - 2:00 PM or Thursday 10:30 - 2:00 PM or Friday 10:30 - 2:00 PM.

Hi @scarlett2018 , just a follow-up, would there still be a meeting sometime this week?

Hi @pw2393, sorry for the delayed responds, this week actually not working well for all the reviewers. I started a doodle vote for next week, and there are currently 4 options, hope some of them works well for you:
https://doodle.com/poll/5ky9scc4xmb3mkh7

Thanks @scarlett2018. Slot is selected. Could you send us the meeting invite via email?

Thanks @pw2393, this Wed does not work out in the end, I pick up a same time slot for next Wed and send out the invitation, hope it work for you.

Thanks @scarlett2018, the mail invite is accepted.

@SparkSnail

Please add a AdlMode.md doc under https://github.com/microsoft/nni/tree/master/docs/en_US/TrainingService, give a description for the training service and explain how to prepare environment and start experiments.

Yup, we are working on it.

@SparkSnail done here:  https://github.com/microsoft/nni/blob/7a73286699a4da148ec7319763ba472a147470c8/docs/en_US/TrainingService/AdaptDLMode.md

I'll help to add an adl pipeline and help to test this code in pipeline. after the pipeline is ready, this pr can be merged.  Will give a notification when I have any progress.


jupyter extension - fix for setting address not returning to the config screen

INT-1230

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Kevin Yang seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Z-Way  - my first "Hello Homekit"

@SphtKr: I've plug in a RazBerry into one RPi2 (BTW running node 4.1.1) and started with Z-Way.
It did the job at first go, awesome. Thanks.
My first included device is a Fibaro FGS222. It works properly but in the Z-Way Expert UI > Configuration > Association I've got an error (warning):
 Device changed configuration but some values are not written. Wait for device to wake up.

Could you please indicate a source (link) where I can investigate about this issue?

Sure, you may want to ask over at the RaZBerry forums, as I'm guessing this isn't specific to Homebridge--do you see the same kind of error when using the device via the ZWay web UI (e.g. http://your.rpi.ip:8083/ )? And/or does it work properly when you control it there?
Just off the top of my head that message might indicate a reception/range problem between the Fibaro and the RPi (which would be easy to happen with just one device), but I don't think I've ever seen that error specifically.

Thanks, I know it isn't a homebridge issue, actually it works perfectly with homebridge as well as with UI port 8083 but this message irritates me. Let see what I can find on the internet.

Ah. For what it's worth, I checked one of my Fibaro dimmers and I have the same message. My guess is that this has something to do with the infamous "Lutron Patent" workaround used by many Z-Wave devices, which uses an association group to report status changes--perhaps Z-Way and the Fibaro dimmer have a problem in the implementation.
I imagine this will be zero-impact--I only noticed the message because of this thread and it's probably been there since I installed that device six months ago. But yes I can see how it would be annoying once you've noticed it.

... so far, it must be an issue with Group 3, I removed all Association and added Group 1, 2 and 3.
Group 1 and 2 are OK, only Group 3 shows a warning.

As this isn't a homebridge issue I'm closing this for now.


Incorrect definition of the system console encoding in Windows used by cmd.exe

Hello.
Trying to compile a simple script (encoding - utf-8), got error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf4 in position 41: invalid continuation byte::
scons: Reading SConscript files ...
SCONS:  Tools= ['mingw']
Scons CC 'gcc' version check:
Scons CC 'gcc' version check gives '5.3.0' from '5.3.0'
Initial CC: gcc
Initial CCVERSION: 5.3.0
Initial CXX: g++
Initial CXXVERSION: None
SCONS:  Picked compiler: gcc
SCONS:  Picked name: gcc
Scons compiler: Using H:\Programs\Miniconda3\Library\mingw-w64\bin\gcc.exe
Scons: Detecting arch with : ['objdump', '-f', 'h:\\Programs\\Miniconda3\\python.exe']
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf4 in position 41: invalid continuation byte:
  File "H:\Programs\Miniconda3\lib\site-packages\nuitka\build\SingleExe.scons", line 1387:
    if getLinkerArch() != getCompilerArch():
  File "H:\Programs\Miniconda3\lib\site-packages\nuitka\build\SingleExe.scons", line 1363:
    _linker_arch = _getBinaryArch(os.environ["NUITKA_PYTHON_EXE_PATH"])
  File "H:\Programs\Miniconda3\lib\site-packages\nuitka\build\SingleExe.scons", line 1345:
    data = data.decode(sys.stdout.encoding)

Some information about environment:
Machine configuration: Windows 10 x64 (Russian language).
Python 3.6.7 installed from Miniconda 3, Nuitka DEV installed from Factory (pip install -U "http://nuitka.net/gitweb/?p=Nuitka.git;a=snapshot;h=refs/heads/factory;sf=tgz").
h:\Programs\Miniconda3>python -m nuitka --version
0.6.4rc3
Python: 3.6.7 |Anaconda, Inc.| (default, Oct 28 2018, 20:57:12) [MSC v.1915 32 bit (Intel)]
Executable: h:\Programs\Miniconda3\python.exe
OS: Windows
Arch: x86

h:\Programs\Miniconda3>python.exe
Python 3.6.7 |Anaconda, Inc.| (default, Oct 28 2018, 20:57:12) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import locale
>>> print(locale.getpreferredencoding())
cp1251
>>> print(sys.getdefaultencoding())
utf-8
>>> print(sys.stdout.encoding)
utf-8
>>> exit()

h:\Programs\Miniconda3>chcp
Текущая кодовая страница: 866

This error is a child of Windows codepage hell.
Python get preferenced encoding cp1251, default encoding utf-8, stdout encoding utf-8 a-a-and windows console encoding is cp866! Bingo!
So we have an incorrect definition of the system console encoding in Windows used by cmd.exe.
If i change line 1345 of \lib\site-packages\nuitka\build\SingleExe.scons from
data = data.decode(sys.stdout.encoding) to
data = data.decode('cp866'),
all works perfect, and compilation finishing successfully.
Problem is to detect correct codepage used by cmd.exe.

This only happens with MSVC, which it seems will output without respect to anything, in whatever encoding the compiler language has. I have no idea how to predict the encoding that MSVC will use. The only approach currently would be to uninstall the languages, but I believe this may also be interface language for Visual Studio.
The sys.stdout.encoding is a failed attempt indeed, because MSVC will not respect it. I wonder if there is any way to get it from some form of --version output. Can you provide a sample of your cl.exe --version as a binary file (i.e. redirected to a file), so I have something to check on.

Can you provide a sample of your cl.exe --version as a binary file (i.e. redirected to a file), so I have something to check on.
Here it is. And yes, this file have OEM866 codepage.
cl-output.txt

@omgiafs
About compilation without VS - I installed mingw64 (it quite small) - after that you can use  --mingw64 in command line

Nuitka now downloads the winlibs compiler automatically and uses it. That should solve the issue for you. For decoding outputs, we now have code that attempts many things. Since this is used, never heard of it again:
def decodeData(data):
    """Our own decode tries to workaround MSVC misbehavior."""
    try:
        return data.decode(sys.stdout.encoding)
    except UnicodeDecodeError:
        import locale

        try:
            return data.decode(locale.getpreferredencoding())
        except UnicodeDecodeError:
            return data.decode("utf8", "backslashreplace")


usbipd install

usbipd should be installed via msi instead of winget

If I understand correctly, the WSL USB GUI MSI will install the correct version of USBIPD without needing to install USBIPD manually.  This should prevent version conflicts.

To clarify, I think that instructions for installing WSL USB GUI replace both of these sections:
https://github.com/intercreate/docs/blob/24c024f0ac5618109d035923b43e36bd4bf50434/Windows/wsl2.md?plain=1#L100-L118
Still good to link to the USBIPD github and the Microsoft article in case someone wants to setup USB forwarding without the GUI.

@JPHutchins


Compose component doesn’t grow to fit the Communication is barred text on mWeb

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open app
Navigate to any chat
Enter multiline message

Expected Result:
Compose component grows with message
Actual Result:
Compose component doesn’t grow to fit the Communication is barred text on mWeb
Workaround:
unknown.
Platform:

Where is this issue occurring?

Mobile Web

Version Number: 1.1.33-0
Reproducible in staging?: Y
Reproducible in production?: Y
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL:
Issue reported by: @puneetlath
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1642722558251700
View all open jobs on GitHub

Triggered auto assignment to @stitesExpensify (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

Should be a simple style update. Marking external

Triggered auto assignment to @mallenexpensify (External), see https://stackoverflow.com/c/expensify/questions/8582 for more details.

Triggered auto assignment to Contributor-plus team member for initial proposal review - @rushatgabhane (Exported)

Current assignee @stitesExpensify is eligible for the Exported assigner, not assigning anyone new.

Looks like you're 'stuck' on this issue @stitesExpensify since you didn't unassign after the Engineering review.  :)
Job posted https://www.upwork.com/jobs/~01b040d9ed08b6e646

PROPOSAL
I think the easiest fix for this one would be renaming the placeholder for something equivalent with length lower than Communication is barred like Unable to chat to english and chat indisponible to spanish.

Proposal
We can make it same look in every size with flex. We can do same thing to Send Button to fill height.
Here is what I'm talking about.

Or only for attachment button,

@mateusbra @okansahin Appreciate your proposals :)
But this is the outcome that we're aiming for. Lemme know if you update your proposals to achieve this. Thanks!

@rushatgabhane I'm living some troubles to got the issues in this repo I think. :)
It's already like the image you have placed. (on iPhoneSE 1st generation.)  I'm confused, please help me.

@okansahin you're right, I can't reproduce this on main either.
[Not urgent] @puneetlath can you please let us know if you can still reproduce this on main since you reported the issue.

Hm, yeah I can't either. I'll close. Thanks y'all!


Separating Database from Application is giving Issues.

We currently use tenancy package for one of our application and we would like to separate our database server from our Application but Hyn wont allow that. Something like https://prnt.sc/t4d1g6. Is there a work around for this?
Thanks in advance .

I had similar problem. Migration is filing because HNY is creating the tenant user with the IP address of the database server causing a permission issue. HNY should create user as 'user@%'

Apologies for the delay.  This has been asked and answered a few times now (here's one example #932, though not a thorough explanation there).  Hyn supports it, but with an extra step.  Not great, but we're not putting a ton of effort into hyn right now.

I added some sample code here: https://github.com/tenancy/multi-tenant/pull/932


Translate Mango Python integration tests to Elixir

Currently, Mango has its own integration tests that are implemented in Python and have to be activated separately via the mango-tests make(1) target.  Re-write them in Elixir to integrate them better with the rest of the integration tests that are in use for CouchDB.  This move could bring more simplification and regularization to the code base, removing the dependency on Nose2.

Good idea. It could be done step-wise, one file or feature at a time. Move from one test suite to the other over time. We did it similarly with Javascript -> Elixir move a few years back.
If bringing in Nose2 is a hassle, another way could be to try to see if Python 3's built-in https://docs.python.org/3/library/unittest.html#module-unittest has the feature nose2 uses. The tests were written during the Python2 days and Python had some improvements since.

Personally, I do not have any problems with Nose2.  But if the integration tests were migrated to Elixir, there is no need for it any more.  On its web site, it considers itself obsolete -- but instead of moving to another Python-based framework, consider Elixir directly.

Personally, I wouldn't write (or migrate) any test to Elixir, because I see no benefits in learning and using a second/third programming language to add contributions to CouchDB. As a new developer to CouchDB (and Erlang), I found using a second language more of an obstacle than an asset.

I am not sure if I got you right @big-r81.  Elixir has already been used in CouchDB and there are a lot of integration tests written in that language.  And, on the top of that, there is Python for Mango.
Or are you saying that we should just leave Python there, because the chances are likely that new contributors are already familiar with it?

@pgj yes, there are a lot of Elixir tests (from the migration from Javascript, if I get this right). This is only a personal preference of me, not to use Elixir for tests. I think the move to using Elixir was made back then to attract more developers. I don't think this happened...
But feel free to use Elixir. 🙂

From my experience as an Elixir test suite contributor, I found that Elixir allows new contributions much easier than Erlang does.
In this case, Python is the exception in the test suite, so I think it is a good idea to simplify this part and migrate these tests to the integration test suite based on Elixir.


cloudflare_tunnel_route flapping due to tunnel id changes

Confirmation

[X] My issue isn't already found on the issue tracker.
[X] I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version
Terraform v1.1.9

provider registry.terraform.io/cloudflare/cloudflare v3.14.0

Affected resource(s)
cloudflare_tunnel_route
Terraform configuration files
# A Named Tunnel resource called zero_trust_ssh_http
resource "cloudflare_argo_tunnel" "tunnel" {
  account_id = data.cloudflare_zone.zone.account_id
  name       = "${local.name_prefix}-cloudflare-tunnel"
  secret     = random_id.tunnel_secret.b64_std
}

# Each route is the CIDR block we will route throught the Cloudflare tunnel
resource "cloudflare_tunnel_route" "route" {
  for_each   = var.tunnel_routes
  account_id = data.cloudflare_zone.zone.account_id
  tunnel_id  = cloudflare_argo_tunnel.tunnel.id
  network    = each.key
  comment    = "New tunnel route for ${each.key}"
}

Debug output
Let me know if you want the debug from the original apply or the subsequent plans that flap
Panic output
No response
Expected output
After doing an apply the tunnel and the routes are created:
module.cloudflare_tunnel.cloudflare_argo_tunnel.tunnel: Creation complete after 1s [id=cffe6b5a-fb67-47ab-9eca-fd44d7f9fc4e]
module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"]: Creating...
module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"]: Creating...
module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"]: Creating...
module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"]: Creation complete after 1s [id=<IP_ADDRESS>/18]
module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"]: Creation complete after 1s [id=<IP_ADDRESS>/18]
module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"]: Creation complete after 2s [id=<IP_ADDRESS>/18]

Doing another plan should generate no changes
Actual output
Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"] has changed
  ~ resource "cloudflare_tunnel_route" "route" {
        id         = "<IP_ADDRESS>/18"
      ~ network    = "<IP_ADDRESS>/32" -> "<IP_ADDRESS>/32"
      ~ tunnel_id  = "8283d830-9303-4a69-a0a8-f2dd01b06f34" -> "74bbf7aa-db68-4763-8b23-ef06ae6b3180"
        # (2 unchanged attributes hidden)
    }

  # module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"] has changed
  ~ resource "cloudflare_tunnel_route" "route" {
        id         = "<IP_ADDRESS>/18"
      ~ network    = "<IP_ADDRESS>/32" -> "<IP_ADDRESS>/32"
      ~ tunnel_id  = "74bbf7aa-db68-4763-8b23-ef06ae6b3180" -> "8283d830-9303-4a69-a0a8-f2dd01b06f34"
        # (2 unchanged attributes hidden)
    }

Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"] must be replaced
-/+ resource "cloudflare_tunnel_route" "route" {
      ~ id         = "<IP_ADDRESS>/18" -> (known after apply)
      ~ network    = "<IP_ADDRESS>/32" -> "<IP_ADDRESS>/18"
      ~ tunnel_id  = "74bbf7aa-db68-4763-8b23-ef06ae6b3180" -> "cffe6b5a-fb67-47ab-9eca-fd44d7f9fc4e" # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"] must be replaced
-/+ resource "cloudflare_tunnel_route" "route" {
      ~ id         = "<IP_ADDRESS>/18" -> (known after apply)
      ~ network    = "<IP_ADDRESS>/32" -> "<IP_ADDRESS>/18"
      ~ tunnel_id  = "8283d830-9303-4a69-a0a8-f2dd01b06f34" -> "cffe6b5a-fb67-47ab-9eca-fd44d7f9fc4e" # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.cloudflare_tunnel.cloudflare_tunnel_route.route["<IP_ADDRESS>/18"] must be replaced
-/+ resource "cloudflare_tunnel_route" "route" {
      ~ id         = "<IP_ADDRESS>/18" -> (known after apply)
      ~ network    = "<IP_ADDRESS>/32" -> "<IP_ADDRESS>/18"
      ~ tunnel_id  = "8283d830-9303-4a69-a0a8-f2dd01b06f34" -> "cffe6b5a-fb67-47ab-9eca-fd44d7f9fc4e" # forces replacement
        # (2 unchanged attributes hidden)
    }

Steps to reproduce
Every time I do plan it shows the flapping
Additional factoids
No response
References
No response

This is the same as #1611 actually (I believe) and all related to the upstream cloudflare-go library not properly sending the query parameters to target a specific tunnel route, so it would return with the wrong one.  The upstream library has been fixed and released.  And #1615 bumps the library here.  We're simply waiting for them to release a new version of this provider.

if you remove this from the module and do the three resources individually, do you get the same issue?

providing the full TF_LOG=DEBUG output will help narrow this down as well.

We updated the provider to 3.15.0 and it resolved the issue. Thank you!


PHOENIX-4370 Surface hbase metrics from perconnection to global metrics

PHOENIX-4370 Surface hbase metrics from perconnection to global metrics
Opening this p.r. for the connivence of discussion

Already merged.


This library does not seem to verify hostname

The fromJson function in Response does not validate the hostname attribute, which leaves this library vulnerable to bot attack.

I'm updating the v1.2 branch to include validation of the hostname and other attributes. Hopefully this should fix the issue.


feat(Navisworks): Retry Sending already converted objects. Implements #2668

This is a highly speculative but functional adaptation. If long sends become possible from the server, this may be moot. I am open to rejections on this.
This PR adds a new Ribbon button that resends a commit that has failed to send to completion post successful Conversion. Motivation is centred around long sends being cancelled, which loses potentially long conversions—wasting users' time.
The converted and draft Commit objects are cached on completing the conversion. A successful Transport and Commit nullify these caches.
The Retry Send button is enabled if the Main thread becomes idle and the cached items still exist, indicating either Transport or the Commit Send failed.
When Retry Send completes, it nullifies the caches, as does changing the document or starting a new Send.
It is a little hacky and isn't completely tied into the DUI just yet - the failed send Sets the CancellationToken as cancelled - can research if this can be uncancelled (A newly created ProgressViewModel is divorced from DUI)

Cudos for the detailed message history 🚀


Let iniparser_getstring() not return NULL if there is a section (not key) with specified name

iniparser_getboolean() can segfault on malformed config files. #125
There are similar problem in iniparser_getlongint() and iniparser_getdouble().
iniparser_getstring() returns NULL if there are section(not key) with that name.
The key does not exists, so it's better to return default fallback.
And returning NULL for "string" is not likely to be considered by programmers, which can be security problem.
Fixes #125

Returning "default" instead on NULL in the case of "section" breaks completely functions iniparser_dumpsection_ini() and iniparser_dump_ini() because iniparser_find_entry() will no longer work for sections (line 303 of iniparser.c). Possibly the best we can do is to return "another_one_default" from iniparser_getstring() function in the case of section.

@dofuuz Please have a look at https://github.com/ndevilla/iniparser/issues/125#issuecomment-2002608649

@lmoellendorf
The segfault problem was fixed with later PR #146(merged).
What this PR is different from #146 is, iniparser_getstring() won't return NULL even if the value is NULL(caused by mallformed ini, or by explicitly set with iniparser_set()). Instead of NULL, it will return string "(null)"
If you think it's OK that iniparser_getstring() can return NULL, you can close this PR.
But in my opinion, returning "(null)" string is more safe option.

In my opinion, returning NULL can be potential problem, so returning "(null)" string is more safe option.

Maybe I am missing something, but I do not understand why parsing a returned pointer for "(null)" should be safer than a quick check for NULL.
As far as I know all standard functions which return pointers, return NULL on error.  In my opinion checking returned or passed pointers for NULL is a mandatory thing to do.

@dofuuz Thank you for the clarification.

@lmoellendorf
Please reconsider merging this PR.
Even authors of iniparser missed the null check and made CVE entry (CVE-2023-33461). #125 #144
Users are more likely to miss that too. 😧

Even authors of iniparser missed the null check and made CVE entry (CVE-2023-33461).

The CVE reads [emphasizes by me]:

iniparser v4.1 is vulnerable to NULL Pointer Dereference in function iniparser_getlongint which misses check NULL for function iniparser_getstring's return.

This was in internal bug, which is fixed.

#125 #144 Users are more likely to miss that too. 😧

I get your argumentation. Programmers tend to forget parameter and return value validation.
But I would rather not resort to returning an arbitrary string instead of NULL.

Making an API Hard to Misuse is good principle of making a library.

If so, I would rather suggest an API like this:
/*-------------------------------------------------------------------------*/
/**
  @brief    Get the string associated to a key
  @param[out] val   Pointer to statically allocated character string
  @param      d     Dictionary to search
  @param      key   Key string to look for
  @param      def   Default value to return if key not found.
  @return     0 on success, -1 else

  This function queries a dictionary for a key. A key as read from an
  ini file is given as "section:key". If the key cannot be found,
  the pointer passed as 'def' is returned.
  The returned char pointer is pointing to a string allocated in
  the dictionary, do not free or modify it.
 */
/*--------------------------------------------------------------------------*/
int iniparser_getstring(const char **val, const dictionary * d, const char * key, const char * def);

...or I would return a pointer to a statically allocated character string, denoting an empty string like it is done in protobuf-c. Making this string part of the API allows for a simple pointer check  as used to with NULL.
This will have to be discussed for v5.x.

Returning "(null)" is not so wrong. With common compilers (including gcc, msvc),
printf("%s", NULL);

prints (null) instead of crashing entire program. (Its behavior is undefined in C standard)
So here is my sugesstion (it's modified from the latter option of your previous comment)
extern const char iniparser_null_string[];
/*-------------------------------------------------------------------------*/
/**
  @brief    Get the string associated to a key
  @param    d       Dictionary to search
  @param    key     Key string to look for
  @param    def     Default value to return if key not found.
  @return   pointer to statically allocated character string

  This function queries a dictionary for a key. A key as read from an
  ini file is given as "section:key". If the key cannot be found,
  the pointer passed as 'def' is returned. If value is `NULL`,
  `iniparser_null_string[] = "(null)"` will be returned instead.
  The returned char pointer is pointing to a string allocated in
  the dictionary, do not free or modify it.
 */
/*--------------------------------------------------------------------------*/
const char * iniparser_getstring(const dictionary * d, const char * key, const char * def);

This will allow explicit pointer check.


Fix: Show actual gql errors for Tasks

Closes #236

Would it be possible to make the error more identifiable as an error? Don't other errors pop up as modals or use bounding boxes in the UI already?

@shreddedbacon totally makes sense - updated with error notifications instead

Much better! :partying_face:


Now it works on mobile like a charm, but something happened to mouse version: Left click to pan stopped working and the "control +" message appears always even if not configured

          Now it works on mobile like a charm, but something happened to mouse version: Left click to pan stopped working and the "control +" message appears always even if not configured

Originally posted by @Xiscollado in https://github.com/HassaanAkbar/vue-zoomable/issues/19#issuecomment-2385073769

Please use the latest version. I have separated the ScrollOverlay component.


Update service tags

Update service tags

#sign-off

@malopMSFT : Thanks for your contribution! The author, @jimdial, has been notified to review your proposed change.

@malopMSFT: I'm sorry - only the author of this article, @jimdial, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

#sign-off


Add "Previous" button to data collection tasks

Fixes #1981

previous_button.webm
@gino-m PTAL?

The build is failing due to
    android.view.InflateException
        Caused by: android.view.InflateException
            Caused by: java.lang.reflect.InvocationTargetException at SurveySelectorFragmentTest.kt:246
                Caused by: java.lang.OutOfMemoryError

@gino-m Any ideas if it could be related to any recent changes?

Trying this now, but it seems strange since SurveySelectorFragmentTest only has 238 lines.

I think the issue is that GCB uses N1_HIGHCPU_8 machine which has max 8 GB of ram.

I think the issue is that GCB uses N1_HIGHCPU_8 machine which has max 7.2 GB of ram.
https://cloud.google.com/compute/docs/general-purpose-machines#n1_machine_types

Do these tests complete when run locally? Is it normal for Robolectric tests to require so much RAM, or could something else be going on here?

The tests pass in the local environment only after increasing the memory limits. It seems unusual for unit tests to consume so much memory. I do remember seeing a bunch of warnings related to memory leakage when running the tests at HEAD as well.

The issue is happening even if I remove the newly added unit tests and only keep the "previous" button related changes. So, there is no way to check-in the changes without breaking the build. I'll try understanding how to resolve the memory leakage issue which is causing the heap size to grow so big.

The issue is happening even if I remove the newly added unit tests and only keep the "previous" button related changes. So, there is no way to check-in the changes without breaking the build. I'll try understanding how to resolve the memory leakage issue which is causing the heap size to grow so big.

I see the same when running locally. The error only occurs when running multiple tests, running testing in SurveySelectorFragmentTest pass. Also seeing this error in unrelated tests like com.google.android.ground.ui.signin.SignInFragmentTest.

I was able to generate the hprof file containing the heap dump. Analyzing it now.

I also found that gradle.properties options aren't applied to unit tests. So, slightly increasing the heap size for unit tests should fix the problem. Tried looking at heap but wasn't able to narrow down to a specific issue. In general, a major chuck of memory was taken by shadowVMRuntime which is used by robolectric.


How can I manually start and stop advertising?

Hello all,
I wish to know if it is possible to manually stop advertising a BLE server (without a connection being established). I can start advertising by using:
BLEDevice::startAdvertising();

In my code it looks like this:
BLEAdvertising *pAdvertising = BLEDevice::getAdvertising();
pAdvertising->addServiceUUID(PAIRING_SERVICE_UUID);
BLEDevice::startAdvertising();

This works, but I haven't found a way to stop it without actually connecting to the device. I have tried this function:
pAdvertising->stop();

But no success. Can anyone help?
Thanks in advance.

Hi,
there is small issue in library that is causing autostart advertising. Delete this line:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/BLEAdvertising.cpp#L496

Thank you for the fast response.
I deleted the line, but the issue still persists. In case you want the full sketch, here it is (it's a VERY lightly modified version of your server sketch):
#define PAIRING_SERVICE_UUID "aa11aa11-bb22-cc33-dd44-ee55ee55ee55"
#define CHARACTERISTIC_UUID  "beb5483e-36e1-4688-b7f5-ea07361b26a8"

class MyServerCallback : public BLEServerCallbacks {
  void onConnect(BLEServer* pserver) {
    Serial.println("Connected");
  }

  void onDisconnect(BLEServer* pserver) {
    Serial.println("Disconnected");
  }
};

void setup() {
  Serial.begin(115200);
  Serial.println("Starting BLE work!");

  BLEDevice::init("");
  BLEServer *pServer = BLEDevice::createServer();
  BLEService *pService = pServer->createService(PAIRING_SERVICE_UUID);
  BLECharacteristic *pCharacteristic = pService->createCharacteristic(
                                         CHARACTERISTIC_UUID,
                                         BLECharacteristic::PROPERTY_READ |
                                         BLECharacteristic::PROPERTY_WRITE
                                       );

  uint16_t tempData = 0xAA;
  pCharacteristic->setValue(tempData);

  pService->start();
  // BLEAdvertising *pAdvertising = pServer->getAdvertising();  // this still is working for backward compatibility
  BLEAdvertising *pAdvertising = BLEDevice::getAdvertising();
  pAdvertising->addServiceUUID(PAIRING_SERVICE_UUID);
  pAdvertising->setScanResponse(true);
  pAdvertising->setMinPreferred(0x06);  // functions that help with iPhone connections issue
  pAdvertising->setMinPreferred(0x12);
  BLEDevice::startAdvertising();
  Serial.println("Characteristic defined!");

  //delay(3000);

  pAdvertising->stop();
  BLEDevice::getAdvertising()->stop();

  Serial.println("Adv stopped?");

  pServer->setCallbacks(new MyServerCallback());
}

void loop() {
  // put your main code here, to run repeatedly:
  delay(2000);
}

You can use only one of them:
 pAdvertising->stop();
  BLEDevice::getAdvertising()->stop();

but even if you use both it shouldnt cause any issue in app.

Thanks a lot. I think I got it to work. On another note, I have a small question to ask. It's about the
"getValue()" function in BLECharacteristic (server side). From what I can see it is possible to write different types of values in the characteristic:
	void setValue(uint8_t* data, size_t size);
	void setValue(std::string value);
	void setValue(uint16_t& data16);
	void setValue(uint32_t& data32);
	void setValue(int& data32);
	void setValue(float& data32);
	void setValue(double& data64);

However, getValue() can only return the value of a characteristic as an "std::string" type and nothing else :
std::string    getValue();

If I want to read this value as a hex value (for example), I could try to create a function that converts this string into a hex, but if there are other simpler solutions, I could really use it.
Thanks in advance.

https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/BLECharacteristic.cpp#L191

Thanks! it was a lot simpler than I thought. I will close this issue now.


make hmfc pip installable

https://goodresearch.dev/setup#create-a-pip-installable-package-recommended

or add to each script:
import sys,os
sys.path.append(os.getcwd())


Render embedded code preview by permlink in markdown

Ready for review

Can we also render branch/tag code lines?

It said line 6 - 16, but in fact this file has only 14 lines.

Can we also render branch/tag code lines?

It only supports "Copy Permlink" like GitHub: https://github.com/go-gitea/gitea/blob/main/README.md#L1

It said line 6 - 16, but in fact this file has only 14 lines.
And the indents are not right.

It has been fixed by the latest commit, please take a try.

There is a padding at the bottom and maybe we can have a radius corner for the code block? Otherwise LGTM.

Done in 289479d

One more thing: Can you override this CSS so it's no longer effective? It causes unwanted alternate row coloring.

Could you help to edit this PR directly? I am not sure what you'd like to see.

Pushed it in https://github.com/go-gitea/gitea/pull/30234/commits/8ae1fc7c4a55942a28a4a7f057adec719526fb3b, now there is no more alternate row coloring:

Oh and one more nit: Can we get rid of this unneccesary empty <p> when the comment content is only a link?

Oh and one more nit: Can we get rid of this unneccesary empty <p> when the comment content is only a link?

I have tried but it is quite hacky, so I chose to keep it as-is. The reason is that we can't easily guess what the node element is. For example, sometimes it is a <p>, sometimes it is a <li>, there might be more cases. So I think it could be done in the future with more tests and better approaches.

Hmm I guess we can workaround via CSS :empty to hide it.

I definitely would say no to the :empty solution. It makes the mechanism more complex than it should be.

Hmm I guess we can workaround via CSS :empty to hide it.

I definitely would say no to the :empty solution. It makes the mechanism more complex than it should be.

Added in https://github.com/go-gitea/gitea/pull/30234/commits/ab87eefa1a21f8e43fd4a1cb611af6f3d8f35350. I wonder if it ever makes sense to render empty p in Markdown. I think not. If you have a better fix in backend to avoid rendering it, sure that would be preferred.

If you have a better fix in backend to avoid rendering it, sure that would be preferred.

It is not feasible at the moment, because the backend has another blocker: IIRC removing an empty node will make other processors panic, it needs a big refactoring. So feel free to go with your solution.

FWIW, GitHub also has this empty p bug 😆
https://github.com/silverwind/symlink-test/issues/14#issuecomment-2032576848

I'd be willing to revert if you add a comment in the backend code where the empty p is produced.

Added some comments for the "empty node" problem. IIRC it is so.

Reverted. I don't like the current rendering, but at least it's just as broken as GitHub's rendering is, so in a sense, it's consistent.

Reverted. I don't like the current rendering

Well, I take it back ... after adding more comments. Hopefully it could make the UI feeling better.

And I suggest to backport it to 1.22, because:

1.22-rc0 is far from stable, it still contains a lot of refactorings, and this PR is quite clear/independent and is covered by tests, it shouldn't cause problems.
Some related testing code changes are important for further backporting works.

Could you please give a hint, how to test this feature?

View  a file and "Copy Permlink", and copy the "permlink" to a comment, it is useful for code reference:

Managed to make it work
Requirements that helped

hosted on https
accessed by domain name
If Gitea accessed via http://localip:port - no magic happened.

Oh yes, it should be under the ROOT_URL.  The ROOT_URL problem should have warned site admins many times, see the admin panel ......


Added a short code of conduct

[ x] I have read freeCodeCamp's contribution guidelines.
[x ] My pull request has a descriptive title (not a vague title like Update index.md)
[ x] My pull request targets the master branch of freeCodeCamp.
[ x] None of my changes are plagiarized from another source without proper attribution.
[ x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Thanks, @nikcodes We want people to read our full code of conduct (it's only 196 words) so we should always link to that instead of trying to summarize it.


Transfer current cloud table from standards repo

bump gx-scs v2 -> v3
aov.de -> www.aov.de
move syseleven v4 -> v3

independent on the check failures: I am fine, but as mentioned I find confusing to see few identical badges in a column. Knowing about virtual rows for regions I understand how to interpret it, but otherwise it is hard. And since MD doesn't support merged cells I suggest either adding a text in front of a badge with a region name or include it into the badge title directly

Yes, @gtema this shall be improved, but it's a separate issue.

Oh, and the checks... the Health Monitor for gx-scs still seems to be unreachable. Apart from that, the failures are not due to changes of this PR.


short delay when switching to normal mode

when pressing escape in insert mode it takes a short delay (maybe 200 msec) before you actually enter normal mode. because this interferes with fast navigation I was wondering if this delay can be disabled?

I think the current problem is here and the relevent issue
if somebody want to solve it, it would be awsome

I've rewritten node-mongovi into a pure C variant using libedit so I no longer require readline-vim or nodejs.

you can use escapeCodeTimeout option
const rlv = require('readline-vim')
const readline = require('readline');

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
  escapeCodeTimeout: 50,
});

rl.question('What do you think of Node.js? ', (answer) => {
  // TODO: Log the answer in a database
  console.log(`Thank you for your valuable feedback: ${answer}`);
  rl.close();
});

rlv(rl);

Thanks @raoofha!
How would one integrate that interface into repl? When I do something like below, it's typing in doubles. So if I type 'k; in the console, it would type 'kk'.
I can't find an option in node repl docs on passing it a customer interface.
let repl = require('repl');
let path = require('path')
let os = require('os')
let rlv = require('readline-vim')
const readline = require('readline');

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
  escapeCodeTimeout: 50
});

let replServer = repl.start({
  prompt: 'kwrr > ',
});

replServer.setupHistory(path.join(os.homedir(), '.node_repl_history'), (e) => null)

rlv(rl)
var vim = rlv(replServer)

it's been a long time since I used this library. the problem is in here escapeCodeTimeout never passed to readline interface function. it you make a pull request to node and fix it you should be able to use it like this
let repl = require('repl');
let path = require('path')
let os = require('os')
let rlv = require('readline-vim')

let replServer = repl.start({
  prompt: 'kwrr > ',
  escapeCodeTimeout: 50,
});

replServer.setupHistory(path.join(os.homedir(), '.node_repl_history'), (e) => null)

var vim = rlv(replServer)

Thank @raoofha. I took a different route using rlwrap.
In case anyone is wondering about this in the future something like below might work for you:
NODE_NO_READLINE=1 rlwrap -t dumb --always-readline node <your custom repl script name>


FALCON-2082 Add CSRF filter for REST APIs

Provide the user an option to enable CSRF filter and configure custom header and browser agents. Forked RestCsrfPreventionFilter from Hadoop 2.8.0 and fixed checkstyle errors. Tested with CSRF filter enabled and custom header "FALCON-CSRF-FILTER" that 1) GET methods always work; 2) POST/DELETE methods will be accepted only if the user provides the correct custom header (See picture below); 3) Falcon CLI works well.

+1

Instead of creating a dependency on Hadoop 2.8, we can just copy the same filter (it's just one class for code and one class for tests).

As I had predicted quite some time back, UI specific code has started creeping into our backend (deja-vu @sriksun ? :stuck_out_tongue: ).  In general, I don't like the idea of UI only concerns creeping in REST services, such code are better handled by UI specific frameworks. We already have several commands just for UI and it keeps growing.
May be we should just switch to a more robust UI framework instead of giving one solution at a time and building everything on our own.

Also, this is not the best way to implement CSRF protection. The most recommended way to implement CSRF is the Synchronizer (CSRF) Tokens. Mechanisms like above are not safe against plugin vulnerabilities like this

May be I overlooked something obvious, but how is the filter distinguishing between requests from UI / other API calls?

As I had predicted quite some time back, UI specific code has started creeping into our backend (deja-vu @sriksun ? 😛 ). In general, I don't like the idea of UI only concerns creeping in REST services, such code are better handled by UI specific frameworks.

Not sure how a UI specific framework alone be useful in this case for not adding this filter - There is nothing that prevents a user to have a UI that call the REST api without going through your UI framework

May be I overlooked something obvious, but how is the filter distinguishing between requests from UI / other API calls?
In RestCsrfPreventionFilter, they only perform CSRF check if the request is from the browser.

Instead of creating a dependency on Hadoop 2.8, we can just copy the same filter (it's just one class for code and one class for tests). I would prefer to avoid dependency on Hadoop 2.8
Thanks for bringing this up, @ajayyadava . @vrangan and I had a discussion on this before. Because RestCsrfPreventionFilter can be shared among all Hadoop components, including hive, oozie, hbase, etc., we want to leave the implementation and improvements to this class, instead of reinventing the wheel.

Also, this is not the best way to implement CSRF protection. The most recommended way to implement CSRF is the Synchronizer (CSRF) Tokens. Mechanisms like above are not safe against plugin vulnerabilities like this
Yes, adding synchronizer tokens would definitely make the filter stronger. @vrangan: Do you know if Hadoop team has this plan to improve RestCsrfPreventionFilter? Thank you.

Not sure how a UI specific framework alone be useful in this case for not adding this filter - There is nothing that prevents a user to have a UI that call the REST api without going through your UI framework

If user is using his own UI framework, then security concerns of UI are also his concern and not of the REST API. It will be same for other UI concerns like cookie management etc. They make sense only for UI client and should be part of UI and not REST API. The reason that we have to put them in REST API is because our UI can not handle it on its own.

In RestCsrfPreventionFilter, they only perform CSRF check if the request is from the browser.
From the hadoop patch it seems to me that once this is enabled, it will act on all clients (including cli) and not just UI. My question is how is it being determined that a request is from the browser and not through other client say cli?

Because RestCsrfPreventionFilter can be shared among all Hadoop components, including hive, oozie, hbase, etc., we think it is better to leave the implementation and improvements to this class, instead of reinventing the wheel.

If we were already on Hadoop 2.8 then I would have also agreed. However, currently I am of the opinion that is better to "copy the wheel" and allow existing falcon users to use this feature without requiring them to upgrade to next version of Hadoop.

If you looked at the code of RestCsrfPreventionFilter line 112-119, they only filter if the request comes from a browser. We can also configure the browser types that need the filter (I'll add this option to the patch in addition to the customized header as I mentioned in my previous reply). I also tested that Falcon CLI works well with this filter.
{code}
public void handleHttpInteraction(RestCsrfPreventionFilter.HttpInteraction httpInteraction) throws IOException, ServletException {
if(this.isBrowser(httpInteraction.getHeader("User-Agent")) && !this.methodsToIgnore.contains(httpInteraction.getMethod()) && httpInteraction.getHeader(this.headerName) == null) {
httpInteraction.sendError(400, "Missing Required Header for CSRF Vulnerability Protection");
} else {
httpInteraction.proceed();
}
}
{code}

Good point. I'll fork the existing filter to Falcon so we avoid Hadoop upgrade issues. In the future, we need to manually sync up with their improvements if any.

@yzheng-hortonworks Yes that is what I was expecting to see in the patch but I couldn't find. I was looking only at the issue that you provided in the JIRA HADOOP-12691 and that doesn't contain this code. Turns out, it is part of the related JIRA HADOOP-12758. Thanks I will review again.

Ok. Thank you @ajayyadava . I have forked RestCsrfPreventionFilter from Hadoop 2.8.0 and also improved code structure. Going to add options in startup.properties for custom header and browser user agent.

Added configuration properties to allow custom header and browser agent. Also fixed checkstyle errors in Hadoop 2.8.0 RestCsrfPreventionFilter. Tested with CSRF filter enabled and custom header "FALCON-CSRF-FILTER" that 1) GET methods always work; 2) POST/DELETE methods will be accepted only if the user provides the correct custom header. See picture below; 3) Falcon CLI works well. @vrangan @ajayyadava  Could you review? Thank you!

I think the option to integrate the Hadoop 2.8 code into the filter here with the config options is good to provide the basic CsrfProtection wihtout having hadoop 2.8 dependencies.
+1 for the options also

I am away from my system for another 20 days and won't be able to review it. From the comments it seems all my concerns are addressed.
Request any one of the committers to review and commit it.

+1. LGTM!


Other Pre-trained Models

Hi,

Can you share your other pre-trained models?
What does mean 'DAVIS17-motion_test_adap_with_crf' and 'DAVIS17-motion_test_adap_without_crf' what is the difference with 'DAVIS17-motion_flow_only'

Thanks for your interest in our work.

The OCLR model is synthetically trained (therefore only one pretrained model) and can be applied to different datasets.

The results with "test_adap" means using the DINO mask propagation method (folder dino/) to refine the flow-only predictions, therefore RGB information is involved in this process.
In detail,
"DAVIS17-motion_flow_only": Predictions from flow-only OCLR model;
"DAVIS17-motion_test_adap_without_crf": Predictions from flow-only OCLR model + DINO-based test time adaptation
"DAVIS17-motion_test_adap_with_crf": Predictions from flow-only OCLR model + DINO-based test time adaptation + an extra CRF process


Error creating windows installer with jPackage

On running mvn clean install  the build works then hits an error when running the jpackage-win-installer phase. The error it gives out is yielded below:
Bundler EXE Installer Package skipped because of a configuration problem: java.lang.module.FindException: Error reading module: C:\Users\shadrach PC\Documents\programming projects
\Maven\clip_share\target\modules\clipShare.jar
Failed to execute goal io.github.wiverson:jtoolprovider-plugin:1.0.23:java-tool (jpackage-win-installer) on project clipShare: jpackage 1 -> [Help 1]
Is there anything cause for this ?

There is a property in the build script,  that should match the name of the artifactId but with dots instead of dashes.
In the template as it ships, the artifactId is...
<artifactId>maven-jpackage-template</artifactId>

...and the java-mod-name property is set to...
maven.jpackage.template
If you change the name of your artifact to, say, clipshare, than you should change the name of the java-mod-name to match, as in...
clipshare
...and...
clipshare
I would strongly recommend sticking to lower-case only names - some file systems are case dependent, some are case independent.
Let me know if this works!
I'm going to leave this issue open, as I think I can do some stuff to simplify this in the future.

There is a property in the build script,  that should match the name of the artifactId but with dots instead of dashes.
In the template as it ships, the artifactId is...
<artifactId>maven-jpackage-template</artifactId>

...and the java-mod-name property is set to...
maven.jpackage.template
If you change the name of your artifact to, say, clipshare, than you should change the name of the java-mod-name to match, as in...
clipshare
...and...
clipshare
I would strongly recommend sticking to lower-case only names - some file systems are case dependent, some are case independent.
Let me know if this works!
I'm going to leave this issue open, as I think I can do some stuff to simplify this in the future.

I tried renaming all to lowercase still the issue persists.
The structure of source folder is show below

Does the structure of the project have anything to do with the issue at hand ?

I tried renaming all to lowercase still the issue persists.
The structure of source folder is show below

Does the structure of the project have anything to do with the issue at hand ?

Hmm. The error message is saying that jpackage can't read the module declared in clipShare.jar. That means either it can't find the path (perhaps the space in the name) or there is something wrong with the jar file (perhaps it's missing some configuration info). The jar file in the template project is generated by the moditect-maven-plugin.
Does just checking out a copy of the template, perhaps in a C:\src\test folder with no names and running it work for you?

Hmm. The error message is saying that jpackage can't read the module declared in clipShare.jar. That means either it can't find the path (perhaps the space in the name) or there is something wrong with the jar file (perhaps it's missing some configuration info). The jar file in the template project is generated by the moditect-maven-plugin.
Does just checking out a copy of the template, perhaps in a C:\src\test folder with no names and running it work for you?

I removed the spaces and yet still did not work.
I cloned the template and tried with your source code and it worked with your codes.
I am starting to feel it's from the structure of my project or probably me having a resource folder, or using fxml instead of building the UI with java or probably it's from the libraries I used ?

I removed the spaces and yet still did not work.
I cloned the template and tried with your source code and it worked with your codes.
I am starting to feel it's from the structure of my project or probably me having a resource folder, or using fxml instead of building the UI with java or probably it's from the libraries I used ?

It's possible it's a matter of sorting out JavaFX module configuration. jpackage indicating that it can't read the module is the key, of course.
Are you doing man clean install or just install?  Try doing clean for each run first, see if that works, or at least gives you a different error...?

It's possible it's a matter of sorting out JavaFX module configuration. jpackage indicating that it can't read the module is the key, of course.
Are you doing man clean install or just install?  Try doing clean for each run first, see if that works, or at least gives you a different error...?

Yeah, if you are using FXML you need to have the javafx.fxml mod loaded. Look for every place that lists the other javafx modes (e.g. javafx.base, javafx.controls) and add javafx.fxml.
Try adding javafx.fxml to the modules-in-use property line like this:
<modules-in-use>javafx.base,javafx.controls,javafx.graphics,javafx.fxml,java.logging</modules-in-use>

Yeah, if you are using FXML you need to have the javafx.fxml mod loaded. Look for every place that lists the other javafx modes (e.g. javafx.base, javafx.controls) and add javafx.fxml.
Try adding javafx.fxml to the modules-in-use property line like this:
<modules-in-use>javafx.base,javafx.controls,javafx.graphics,javafx.fxml,java.logging</modules-in-use>

As an FYI, here are all the javafx jmods:
javafx.base.jmod	javafx.fxml.jmod	javafx.media.jmod	javafx.web.jmod
javafx.controls.jmod	javafx.graphics.jmod	javafx.swing.jmod

As an FYI, here are all the javafx jmods:
javafx.base.jmod	javafx.fxml.jmod	javafx.media.jmod	javafx.web.jmod
javafx.controls.jmod	javafx.graphics.jmod	javafx.swing.jmod

Finally the the native installers is built successfully. The error was coming from the META-INF file. By removing it, the build ran successfully.

Finally the the native installers is built successfully. The error was coming from the META-INF file. By removing it, the build ran successfully.


few typo fixes

+1
In case someone on the team wants to review or merge it.

+1

commit df7421c thanks.


Duplicate commits in diagram

First two commit tree diagrams at http://git-scm.com/2010/03/02/undoing-merges.html have two commits called C8 and none called C9. Unfortunately part of the description is to "rewind master back to C8" so although it should be clear that it's the C8 on master the text is on about, it may well confuse new users.

duplicated from https://github.com/git/git-scm.com/issues/207
@peff close?


fix initiliazed => initialized

fix initiliazed => initialized
Signed-off-by: Guanqi Bing<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Thanks for your contribution.  You can check here.

Codecov Report

Merging #762 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #762      +/-   ##
==========================================
+ Coverage   46.55%   46.56%   +0.01%
==========================================
  Files         113      113
  Lines        6352     6352
==========================================
+ Hits         2957     2958       +1
  Misses       3156     3156
+ Partials      239      238       -1

Impacted Files
Coverage Δ

dfget/core/uploader/peer_server_executor.go
91.52% <100%> (ø)
:arrow_up:

supernode/daemon/mgr/scheduler/manager.go
24.81% <0%> (+0.75%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37dd54b...d35a84c. Read the comment docs.

LGTM.


fix(ios): Do not crash when accessLog return nil

Summary
accessLog method can return nil if no logging information are currently available (see https://developer.apple.com/documentation/avfoundation/avplayeritem/1388499-accesslog).
Motivation
should fix https://github.com/react-native-video/react-native-video/issues/3424
Changes
I handle the case where accessLog return nil & do not call onVideoBandwidthUpdate if it's nil. That should avoid some crash.
Test plan
Sadly I can not reproduce the crash. I did not find a way to trigger onVideoBandwidthUpdate. Someone maybe can point me a way to reproduce this beahavior ?

I have tested his branch and my issue from #3424 is not freezing anymore.
Great job @gkueny !

@KrzysztofMoch thanks for the review.

I did not find a way to trigger onVideoBandwidthUpdate. Someone maybe can point me a way to reproduce this beahavior ?

For my own knowledge, do you know how a can trigger AVPlayerItemNewAccessLogEntry notification ?


is support registry v2 for now?

There's a PR open (#852) that adds V2 registry support, this is being reviewed.


How are Semantic Kitti sequences 11-21 tested?

Thank you for your great work. I achieved 68.587% mIou (num_vote=1) on validation set 08 using a pre-trained model, and then I want to test sequence 11-21, how do I do this?

You can use --submit_to_server during the testing.

Hello!
thank you for answer, in this case we can't calculate the accuracy beacause we don't have the ground truth , isn'it ?

@brahami14 Yes, you need to submit the prediction results on the official server.


New 3D import: Scene Importer doesn't generate static mesh when re-importing

Godot version
v4.0.dev.custom_build [60dcc4f39]
System information
Windows 10
Issue description
Using the recent scene importer made by reduz, using "Mesh + Static Collider" on a mesh doesn't generate a staticbody as a child, it doesn't do anything or failed to do so, it doesn't look like a regression but something that never worked to begin with.
"Rigid body + Mesh" reset the scale of the root, the created RigidBody didn't get the transform correctly.
Tried on the latest branch 60dcc4f39c0e6289dba691225c002cd6e77be6ba and the one when the importer was first merged(97a3a6622).
I looked at the code source I think the issue is in "editor/import/ressource_importer_scene.cpp" line 624, the way it add a child or set the variable "base", mess up the process of creation of the StaticBody3D.
Steps to reproduce

go in blend_fille, and double click on test_exported_file.gltf, the importer should open, click on the mesh and in generate/physics select "Mesh + Static Collider" and click on re-import.
Minimal reproduction project
IslandDemo.zip

Fixed by by https://github.com/godotengine/godot/pull/50797?


💄(frontend) create form object styles and related components

Purpose
With Joanie, we need form to create / update some data. In order to do that, we decided to create a set of reusable form components.

Proposal

[x] Create form styles
[x] Create a form scheme within settings/_colors
[x] Create reusable React form components

I think the help/error messages are too small and might be inaccessible.
I'm not sure if it's a problem for RGAA compliance or not but it looks like it could be difficult to read for some users.
Other than that I just had a small nitpick with comments

@mbenadda You're right, there is some accessibility issues that I try to fix.

Fix RGAA color contrast issue
Upgrade message font size to improve readability
Add icon next to message to precise context (error or information) for color blind.


[SPARK-8400] [MLLIB] Allow ml.ALS to use -1 block size to auto-configure

If a param value of -1 is entered for user or item block size, the value will be auto-configured during training based on the level of parallelism.

Hi @mengxr, I am proposing to add eq and or methods in ParamValidator to allow for the block size to be set (>= 1 or -1), see inline comment below.

Can one of the admins verify this patch?

@BryanCutler We should keep the changes minimal for bug fixes. I thought about the logic again. If we set numBlocks based on the input number of partitions when numBlocks is -1, we basically assume that users know how to control the number of RDD partitions, while is often not true. So a quick fix would be require(numBlocks > 0) in the branch-1.3 branch. Branch-1.4 already handles that. Sorry about going back and forth! Could you submit a new PR to branch-1.3 and add this check? If you feel or is also useful for other cases, we can create a new JIRA for it.

Sure, I can do that.  I don't think the addition of an or validator would be all that useful, only in special cases like this.

@mengxr, using branch-1.3 I gave ALS a -1 block size and got the following exception:
[info] ALSSuite:
[info] - more blocks than ratings *** FAILED *** (1 second, 112 milliseconds)
[info]   java.lang.IllegalArgumentException: requirement failed: numBlocks must be positive but found -1.
[info]   at scala.Predef$.require(Predef.scala:233)
[info]   at org.apache.spark.ml.recommendation.ALS$LocalIndexEncoder.(ALS.scala:1164)
Seems like it's already fixed to me, what do you think?


Video preview not working

I have the latest version of ArkOS running on a RG351P. Midusage my device turned off due to battery level. After charging the device, the OS would not boot up. I reformatted my SD card and reflashed ArkOS and did a clean install. Now video previews only work for 2 systems despite having all of the images and videos scraped for everything. Box art shows fine, it seems to be only for video preview. I transferred an identical copy of my SD card during the reinstall process and did not alter any settings. Is this a known issue (i could not find it there) and if not, could it be as simple as the proper file directory is not specified?

Sounds like the system can’t find the videos for your other systems.  Double check that your systems have a video folder for each of them within their respective rom folders.  If so,  You can either manual check the gameslist.xml file for each system that’s not functioning and verify that they’re pointing to the right video folders or rescrape the systems that are not playing videos.

That's exactly it. It appears, somehow, all but few of my gamelist.xml files no longer have a video path specified. I now realize this wasn't a ArkOS issue but I appreciate the help on this, I would have been running in circles.


🍒 [5.7] DiagnosticEngine: Print any in 'aka' types

Description: We ought to be passing the same options we use for printing type arguments. Besides any, doing so also mirrors type qualification and inclusion of function convention attributes in 'aka' types, which makes sense.
Risk: Very low.
Tests: Updated existing tests.
Original PR: #59147

@swift-ci please test


Ability to disable Lint/Syntax from extension

Is your feature request related to a problem? Please describe.
I am working on a tool to convert Ruby to Crystal via a rubocop extension, which will inevitably produce invalid Ruby code:

I know this code is invalid Ruby, and I would like to ignore this error.
Describe the solution you'd like
The ability to in some way disable Lint/Syntax from an extension, in the code of the extension, i.e. not building a local modified copy of rubocop with a gutted Lint/Syntax.
Describe alternatives you've considered

I have attempted to disable Lint/Syntax through configuration, which failed as expected.
I've tried to undefine the Lint/Syntax cop in the extension, no luck.
I've tried to create my own Lint/Syntax cop which does nothing, but it seems to have no effect.

While a syntax error triggers the Lint/Syntax, no other cops can operate, making Lint/Syntax the only reliable resource:
https://docs.rubocop.org/rubocop/1.64/configuration.html#enabled
Although Crystal's syntax resembles that of Ruby, RuboCop is a linter designed for parsing Ruby syntax. Therefore, it would be inappropriate to provide an API to disable the fundamental Lint/Syntax.


Tracking: Long haul testing with TB size of table/MView (batch TPCH).

Description:

Create MViews as select all from kafka sources to load data into storage.
Run batch queries, test stability, enhance observability, locate the bottleneck.

Tracking issues:

[x] #691
[x] #1236
[x] Scan optimization (revisit metrics cost and location) (#1421).

Planned works done. Closing this.


Simplify installed property in tsd.json

It would be nice if installed property in tsd.json could be simplified like in bower.json or package.json. The main reason in favor of this style is to provide the following functions:

Ability to provide autocomplete for definitions and versions from IDE/editor (like this: https://visualstudiogallery.msdn.microsoft.com/65748cdb-4087-497e-a394-2e3449c8e61e)
Ability to edit file directly by user
Simplicity to recognize which version is installed

+1: I'd like to see the ability to mark a version (git tag or version read from package.json) and not only a commit uuid. Or is this already possible? Can't seem to find what are the valid entries in the installed hash...

Or is this already possible

Nope. Just git hashes atm


[CONTROLLER/RECORDER] support updating subdomain name

This PR is for:

Server

support updating subdomain name
Checklist

[ ] Added unit test.

Backport to branches

main

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.zhengya seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Announcement:  Intel publish white paper on high speed 300 FPS depth capture for D435

Hi everyone,
Intel have published a new white paper document on the subject of high speed 300 FPS depth capture for the RealSense D435 depth camera.
https://dev.intelrealsense.com/docs/high-speed-capture-mode-of-intel-realsense-depth-camera-d435
There is an accompanying article on the official RealSense blog.
https://www.intelrealsense.com/high-speed-mode-at-300-fps-for-depth-camera/

Very cool @MartyG-RealSense !
@RealSenseCustomerSupport Would it be possible to have the 256x144 mode be pixel binned instead of center cropped so it can maintain the wide field of view? Having a high speed wide field of view depth camera would be outstanding.

The article talks about firmware <IP_ADDRESS> but I don't see this available anywhere.

@GOBish  the mention of new firmware versions in documents such as white papers is not always in sync with their listing on the firmware download pages.  Sometimes it takes more time for download links to appear.  Currently, <IP_ADDRESS> is the most recent available firmware.


Can't get amavis up?

Context

The server won't spin up amavis, so I'm getting tons of errors looking like
postfix/smtp[18326]: 5C39A19CC4B<EMAIL_ADDRESS>relay=none, delay=0.18, delays=0.18/0/0/0, dsn=4.4.1, status=deferred (connect to <IP_ADDRESS>[<IP_ADDRESS>]:10024: Connection refused)
I was simply trying to semd an email to the user. /var/log/mail/mail.err is full of lines like:
Mar 14 05:17:19 mail amavis[463]: (!!)TROUBLE in pre_loop_hook: db_init: BDB no dbS: BDB0002 __fop_file_setup:  Retry limit (100) exceeded, File exists. at (eval 92) line 318.
I don't know what file it's referencing.

Expected Behavior

Email should work?
Actual Behavior

amavis down, only getting lots of errors, not sure of issue
Possible Fix

Steps to Reproduce

5:20:56<EMAIL_ADDRESS>mail cat ./docker-compose.yml
version: '2'

services:
  mail:
    image: tvial/docker-mailserver:latest
    hostname: mail
    domainname: smuns.ch
    container_name: mail
    restart: always
    ports:
    - "25:25"
    - "143:143"
    - "587:587"
    - "993:993"
    - "110:110"
    - "995:995"
    volumes:
    - maildata:/var/mail
    - mailstate:/var/mail-state
    - ./config/:/tmp/docker-mailserver/
    - ./letsencrypt/etc/letsencrypt:/etc/letsencrypt
    environment:
    - ENABLE_SPAMASSASSIN=1
    - ENABLE_CLAMAV=1
    - ENABLE_POP3=1
    - ENABLE_FAIL2BAN=1
    - ENABLE_POSTGREY=1
    - SSL_TYPE=letsencrypt
    - ONE_DIR=1
    - DMS_DEBUG=0
    cap_add:
    - NET_ADMIN

volumes:
  maildata:
    driver: local
  mailstate:
    driver: local

5:21:57<EMAIL_ADDRESS>mail ls -la
total 44
drwxr-xr-x  7 root root 4096 Mar 14 05:21 .
drwx------ 27 root root 4096 Mar 14 05:22 ..
-rw-r--r--  1 root root    0 Jan 29 00:35 cat
drwxr-xr-x  4 root root 4096 Mar 14 05:15 config
-rw-r--r--  1 root root  727 Jan 30 03:51 docker-compose.yml
drwxr-xr-x  8 root root 4096 Oct 13 20:51 etc
drwxr-xr-x  4 root root 4096 Oct 13 21:07 letsencrypt
drwxr-xr-x  2 root root 4096 Oct 13 20:51 log
-rw-r--r--  1 root root 5484 Feb 12 06:26 master
-rwxr-xr-x  1 root root 3626 Oct 13 20:43 setup.sh
drwxr-xr-x  2 root root 4096 Oct 13 20:43 ssl

Your Environment

Amount of RAM available: 512 MB

Mailserver version used: Not sure. Did docker-compose down --rmi all;docker-compose up -d mail and still have the problem

Docker version used: Docker version 17.12.1-ce, build 7390fc6

Environment settings relevant to the config: See above

Any relevant stack traces ("Full trace" preferred):
How do I get this?

The amount of ram with clamav needs to be at least 1Gb.
See https://github.com/tomav/docker-mailserver#requirements

I was running this for about four months recently with no issues... Is there anyway I can not use clamav and just get by?

512Mb is to low, maybe you where on the limit.

I'm running into the same amavis problem, and I am certainly not in an under-provisioned memory situation, as my container has 5 GB RAM available to it, with plenty of free memory.
postfix mail.log is filled with hundreds of these:
Mar 25 06:44:44 mailbots postfix/error[18931]: DB87D13E085E<EMAIL_ADDRESS>relay=none, delay=265423, delays=265423/0.01/0/0.02, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to <IP_ADDRESS>[<IP_ADDRESS>]:10024: Connection refused)

After learning that port 10024 is amavis, I checked /var/log/supervisor/amavis.log and it is absolutely filled with these lines:
Suicide () TROUBLE in pre_loop_hook: db_init: BDB no dbS: BDB0002 __fop_file_setup:  Retry limit (100) exceeded, File exists. at (eval 92) line 318.

As with the OP (@marv3lls), this server has been running extremely well for around 4 months (coincidence?), and just stopped working all of a sudden due to this problem.
Here is the memory situation from inside the failed container:
$ sudo docker exec -it mail bash
root@mailbots:/# free -m
              total        used        free      shared  buff/cache   available
Mem:           4952        1667         254           1        3030        2997
Swap:          2919           1        2918


Details about the partitioning rules

Hi,
great work!
I have been looking into the set of partition rules that you use to parallelize the models.
Would you mind explaining the idea behind these specific partitioning schemes? Is there any particular heuristic that you are following? For example, I have notice that you often invert the "mp" and "fsdp" axis for consecutive layers. Is this to minimize communication costs, or is there any other reason?
Thanks a lot!

Thanks for the question! And sorry, your issue slipped past me and I just noticed it. So apologies for the late reply.
The idea behind the mp axis is to minimize communication. So:
[a] = [a1]
         [a2]
where w1 and w2 are each half of the rows of a matrix w each on a separate device, then when we multiply by a vector [x], we will get:
[a][x] = [a1] [x] = [y1]
             [a2] [x] = [y2]
Where the outputs [y1] and [y2] correspond to half of the full activations on each device. Notably we don't have to do any communication here. We would then apply some activation, which is element wise, so this also has no communication.
Then on the second layer, we can split the weight, b, column-wise to get:
[b] = [b1 b2]
multiplying this by y, gives:
                           [y1]
[b][y] = [b1 b2][y2] = (b1)(y1) + (b2)(y2)
thus we need to do 1 all-reduce communication at the end of the forward pass. And if you compute the backward pass you will find that there is also 1 all-reduce on the backward pass. So this pattern of making the consecutive layers alternate for mp minimizes communication by giving 1 communication on the forward pass and 1 on the backward pass.
For fsdp, it doesn't matter what axis you place it on, it's the same communication either way, so we just place it on the other axis that isn't being used for mp.