[Bugfix] Fix potentially unsafe custom allreduce synchronization

FIX #8410 #8404
Use incrementing counter as discussed, but two sets of counter is needed. Also, we avoid incrementing peer counters to avoid round trip latency. Instead, we increment self counter and write to peer counter which only involves single trip latency. Using one set may cause hang (hang can occur on my H100 test machine). Performance improved for 6~7us for two stage allreduce after this PR on H100. The reason is that we replace a __threadfence_system with acquire-release, which has better performance.
 Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Adding or changing kernels
Each custom kernel needs a schema and one or more implementations to be registered with PyTorch.

Make sure custom ops are registered following PyTorch guidelines: Custom C++ and CUDA Operators and The Custom Operators Manual
Custom operations that return Tensors require meta-functions. Meta-functions should be implemented and registered in python so that dynamic dims can be handled automatically. See above documents for a description of meta-functions.
Use torch.libary.opcheck() to test the function registration and meta-function for any registered ops.  See tests/kernels for examples.
When changing the C++ signature of an existing op, the schema must be updated to reflect the changes.
If a new custom type is needed, see the following document: Custom Class Support in PT2.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

@hanzhi713  it's very exciting to see the results! and it seems we can use custom allreduce for larger size tensor now, it is still faster than nccl for much larger sizes.

I will hand it over to @kanghui0204 for detailed review.
also cc @HydraQYH if you are interested

@youkaichao I noticed that for H100 one-shot allreduce is faster than two-shot until 512kb. The previous threshold is set to 256kb which is mainly tuned for A100. I don't have access to A100 and other ampere devices now, so if we want to add different threshold for different devices I will need your help.

This is by no means urgent or necessary since we'll only save a few us for sizes near the threshold.

agree, we don't need to consider it in this pr.
in the future, I do plan to have two streams for cudagraph capture, one for brute-force tuning and hyper-parameter selection, and another for captureing the running kernel with the optimal configuration. i.e. we online search for optimal configurations, and bake the configurations into cudagraph.

This code looks good to me and is similar to what I thought before. I left two comments that are not particularly important. Additionally, I would like to ask if this MR (Merge Request) has been tested on the actual model (in terms of accuracy and performance)? Or has it undergone longtime testing (accuracy, performance, hang issues)? Or was it only tested with unit tests?

@kanghui0204 we have tests for this code. after we confirm the code change, I can kick off the tests.

I'm currently experimenting whether the acquire-release (or a system-wide fence) is really required for two-stage custom allreduce. In two-stage allreduce, the following will happen

stage 1: reducescatter is performed and the result is saved to a temporary buffer
stage 2: allgather is performed by reading peer GPU's temporary buffer.

In theory it's needed because the current GPU may see stale peer GPU's temporary buffer at the second stage because memory is not coherent across devices. However, in practice, the peer GPU's temporary buffer is read only once in the second stage, and therefore it cannot be in the cache, so it must read the "real" value from peer GPU's memory directly. That's why without acquire-release the result is still correct. If you manually populate the cache by reading peer GPU's buffer before the second stage, you can indeed get stale values which partially proved this hypothesis. See this commit: https://github.com/hanzhi713/vllm/commit/4ba295f20f015ce4b067d24dc324bd58fc1a7b73
In addition, even if you prepopulate the cache with stale values, it seems that we can get around this by using a volatile read. It looks like volatile read at sys scope will bypass the cache and read from peer GPU directly, but this is not mentioned in the manual.
The key take away here is, if my hypothesis is correct, we can save 5us across the board by not using acquire-release. We can do it a bit safer by using volatile read during the second stage of the allreduce at virtually no performance cost, because the cache is not populated and peer read is needed anyway if my hypothesis is correct.
cc @youkaichao @HydraQYH @kanghui0204 for a discussion.
Or, we can keep the correct implementation with acquire-release which is the safest option.

I'll discuss and confirm it with gpu memory experts.
my guess: as long as we use volatile read to read from peer's memory, gpu should not cache anything, because it is unsafe, it does not know if the value is written.
supporting evidence: if I map cpu memory to gpu memory using the following code, every time the kernel reads this memory throuth host_data_in_device, it needs to read the value from cpu. it cannot cache because the pcie read does not support cache coherency.
    size_t N_bytes = size_in_bytes * sizeof(uint8_t);
    CUDA_CALL(cudaHostAlloc((void**)&host_data, N_bytes, cudaHostAllocMapped));
    CUDA_CALL(cudaHostGetDevicePointer((void**)&host_data_in_device, host_data, 0));

let me come back when I get the answer.

https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#cache-operators Using ld.cv in stage 2(allgather) may be safer. Because "The ld.cv load operation applied to a global System Memory address invalidates (discards) a matching L2 line and re-fetches the line on each new load." @hanzhi713 cc @youkaichao

Yeah that worked too. I would presume ld.volatile has similar effect as ld.cv but it's not mentioned in the manual.

Hello, @hanzhi713 , I did some testing on your hypothesis.  It seems that ld.volatile and ld.cv instructions meet our requirements. I put the complete experimental screenshots, code, and report here: https://github.com/HydraQYH/peer_access_demo
Then I verified the performance of the fence-free method on H20. It does seem to improve performance. I submitted the code, but I think it needs to be refined. Since I can't submit commits to your code repository, I opened a new PR: https://github.com/vllm-project/vllm/pull/8690
I put the details of the performance improvement in that new PR.
Maybe you can refer to my PR or work directly on it. Thanks a lot. This is my first time submitting code to the vllm community. If I do something wrong, please contact me in time. cc @youkaichao @kanghui0204

Hi all, I met @youkaichao at the pytorch conference and he mentioned problems with the previous all-reduce implementation in vLLM regarding correctness.
I just want to point out something regarding the conversations about ld.cv and ld.volatile in https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#cache-operators:

Cache operators on load or store instructions are treated as performance hints only. The use of a cache operator on an ld or st instruction does not change the memory consistency behavior of the program.

Basically, cache hints have no effect on correctness, so I don't think this is the right thing to do.
Regarding volatile, unfortunately volatile is not particularly well-defined. The only hints that the cuda documentation gives are this paragraph:

The compiler is free to optimize reads and writes to global or shared memory (for example, by caching global reads into registers or L1 cache) as long as it respects the memory ordering semantics of memory fence functions (Memory Fence Functions) and memory visibility semantics of synchronization functions (Synchronization Functions).
These optimizations can be disabled using the volatile keyword: If a variable located in global or shared memory is declared as volatile, the compiler assumes that its value can be changed or used at any time by another thread and therefore any reference to this variable compiles to an actual memory read or write instruction.

(from https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#volatile-qualifier)
and these two paragraphs:

An ld.volatile operation is always performed and it will not be reordered with respect to other volatile operations to the same memory location. volatile and non-volatile load operations to the same memory location may be reordered. ld.volatile has the same memory synchronization semantics as ld.relaxed.sys.

(from https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#data-movement-and-conversion-instructions-ld)

An st.volatile operation is always performed and it will not be reordered with respect to other volatile operations to the same memory location. st.volatile has the same memory synchronization semantics as st.relaxed.sys.

(from https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#data-movement-and-conversion-instructions-st)
Anyway, my point is that you should never need to use the cache hints or volatile to ensure the correctness of your program. I will have to read more to understand what is going on, though.
I think that @hanzhi713's strategy using acquire-release is going in the right direction.

@hanzhi713 then I guess this pr should be good to go? do you have any other suggestions? @HydraQYH @kanghui0204 @galv

Yeah I think we can merge this PR first. Regarding using weaker sync primitives, it's unsafe because no guarantee is made by the memory model, although often in reality the hardware provides stronger guarantee than the memory model. Let's make a different PR for that if we still decide to go the unsafe route.

Yeah I think we can merge this PR first. Regarding using weaker sync primitives, it's unsafe because no guarantee is made by the memory model, although often in reality the hardware provides stronger guarantee than the memory model. Let's make a different PR for that if we still decide to go the unsafe route.

Agree.

Regarding using weaker sync primitives, it's unsafe because no guarantee is made by the memory model, although often in reality the hardware provides stronger guarantee than the memory model. Let's make a different PR for that if we still decide to go the unsafe route.

Sorry for the basic questions, but what "weaker sync primitives" are you referring to?

Regarding using weaker sync primitives, it's unsafe because no guarantee is made by the memory model, although often in reality the hardware provides stronger guarantee than the memory model. Let's make a different PR for that if we still decide to go the unsafe route.

Sorry for the basic questions, but what "weaker sync primitives" are you referring to?

Using ld.volatile.sys as opposed to a release-acquire followed by regular load. Empirically, it works even after polluting cache with stale values, but this is not guaranteed as per the manual.

Got it. Yes, there are times when you can use volatile loads to communicate things, like with NCCL's LL and LL128 protocols. But it is tricky, and requires knowledge of your hardware. In particular, you need to make sure that all data is communicated atomically (both the flag and the data), which depends upon the alignment of your address, as well as details of the PCIe protocol, nvlink protocol, etc.
So I agree with "Yeah I think we can merge this PR first".


Provide a mechanism to specify the type of workload to be executed.

The container can have an argument that can specify, if all workloads or specific workloads can be executed.
For example:
All workloads
--run-wl=all

or a set of workloads
--run-wl=Basic,File

or a specific workloads
--run-wl=Basic/File-SimpleReadWrite

Moved the issue to openebs/test-tools repo: https://github.com/openebs/test-tools/issues/15


NIFI-10404 TailFile processor persistent state not cleaned up

Summary
NIFI-10404
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[x] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[x] Build completed using mvn clean install -P contrib-check

[ ] JDK 8
[ ] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

LGTM
Thanks @krisztina-zsihovszki for this fix!
Pushed to main.


(PUP-5047) POSIX Exec throws error when group has no name

On Macs which are domain-joined, lookups between group ID and name can
fail if the domain isn't accessible at that moment. In these situations,
lookups on files and other objects return a GID for the user that is
already an integer, which is then passed to convert_xid, which tries to
re-lookup that GID and fails, returning an "Invalid group: ###" error.
This PR bails early if the user/group ID is already numeric, and thus
doesn't need to be looked up.

CLA signed by all contributors.


Invoking require throws 'util is not defined'

Using require just yields this:
pryjs> require('fs')
=>  [ReferenceError: util is not defined] ReferenceError: util is not defined
  at Object.SyncHighlight.stringify (/Users/justin/code/testdouble/waterline-parse/node_modules/pryjs/lib/pry/sync_highlight.js:26:16)
  at Object.stringify (native)
  at new SyncHighlight (/Users/justin/code/testdouble/waterline-parse/node_modules/pryjs/lib/pry/sync_highlight.js:20:29)

In order to get parse to work, I have to do global.util = require('util') at some point earlier in my script, which isn't exactly optimal.

thanks! I upgraded it here but it looks like npm publish is down right now! I'll close this ticket once 0.0.9 is up.

This is part of 0.0.10 release on npm now! [:


cj state note working

Hi Hatef,
cj state stopped working for all my jobs. In particular, this happens

Xiaoyan$ cj state 88af1ee2a2a52947fe11762b8c948fc29c87677a --v
sacct: error: slurmdbd: Too wide of a date range in query
pid 88af1ee2a2a52947fe11762b8c948fc29c87677a
remote_account<EMAIL_ADDRESS>1         12278820            Unknown
2         12278821            Unknown
3         12278822            Unknown
4         12278824            Unknown
5         12278825            Unknown
6         12278827            Unknown
7         12278828            Unknown
8         12278829            Unknown

As you can see I get a sacct: error: slurmdbd: Too wide of a date range in query error and all my jobs are listed as Unknown. Do you know what might be happening?

Hi Xiaoyan,
Do you only have 8 jobs in 88af1ee2a2 ?
Sometimes when you have a large number of jobs, Slurm gives some errors.
If not, you should contact Sherlock if this problem continues. It seems its a Sherlock error.
Hatef


Nixos service fixes

@jbgi should these patches be squashed before we merge, or can one invoke bors to do that?

bors r+


Kafka: Reordered segment allocation causes spurious failures

In one of our test clusters we saw Kafka index task failures corresponding to sequence_name_prev_id_sha1 unique constraint violations on the overlord. The query getting run was this one:
            handle.createStatement(
                StringUtils.format(
                    "INSERT INTO %1$s (id, dataSource, created_date, start, %2$send%2$s, sequence_name, sequence_prev_id, sequence_name_prev_id_sha1, payload) "
                    + "VALUES (:id, :dataSource, :created_date, :start, :end, :sequence_name, :sequence_prev_id, :sequence_name_prev_id_sha1, :payload)",
                    dbTables.getPendingSegmentsTable(), connector.getQuoteString()
                )
            )
                  .bind("id", newIdentifier.getIdentifierAsString())
                  .bind("dataSource", dataSource)
                  .bind("created_date", DateTimes.nowUtc().toString())
                  .bind("start", interval.getStart().toString())
                  .bind("end", interval.getEnd().toString())
                  .bind("sequence_name", sequenceName)
                  .bind("sequence_prev_id", previousSegmentIdNotNull)
                  .bind("sequence_name_prev_id_sha1", sequenceNamePrevIdSha1)
                  .bind("payload", jsonMapper.writeValueAsBytes(newIdentifier))
                  .execute();

The constraint violation happens when the (sequence_name, previousSegmentIdNotNull) tuple is not unique. So in particular, it happens when the segment sequence "forks" and goes from the same segment X into two subsequent segments Y and Z. And from looking at the tasks involved, the events that occurred looked something like this:

A Kafka index task allocated some segments and then failed for some reason.
A new task was started up to re-read from the same offset.
The new task also tried to allocate segments, and it worked for a while, but eventually it tried to allocate one in a different order than the original task. This fails and so does the whole task.
Another task starts up, but it fails for the same reason.
The cycle of violence continues ad infinitum.

I think the problem is that the allocation fails if you don't do them in the same order as they were originally done in, but, since #4815 it's no longer guaranteed that two Kafka index tasks, starting from the same partition/offsets, will create segments in the same order (due to the mixing of messages from different partitions being non-deterministic). I believe this is meant to be okay, since the hypothetical two tasks would still end up at the same place, due to the restriction that there is just one segment per interval now.
(Aside: if there were more than one segment per interval, the tasks would not be guaranteed to end up at the same place, since they wouldn't necessarily make the same decision about which rows to put in which segments for the same interval.)
So I am thinking we should:

Confirm that there is meant to only be one segment per interval per sequence now, and add sanity checks to enforce this if they do not already exist.
Confirm there is no longer any need for the previous-segment-id check.
Remove the check: allow a sequence to allocate segments out of order, subject to there being at most one per interval per segment.

/cc @pjain1 @jihoonson

@gianm nice catch! The supposed solution sounds good to me.

I'm working on this and https://github.com/druid-io/druid/issues/5729.


Rename Remove_All_Idenitites_From_Policy_USR.ps1 to Remove_All_Identi…

…ties_From_Policy_USR.ps1

Hi @megamorf, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;


enhancement: new gpliinventory under 10.0.8 not supportet yet if enforce plugin is active

First of all: very great plugin, thank you very much for this.
additional to existing code
		if (class_exists('PluginFusioninventoryCommunication') && strpos($_SERVER['HTTP_USER_AGENT'], 'FusionInventory-Agent_') !== false){
			if(strpos($_SERVER['REQUEST_URI'], '/plugins/fusioninventory/') !== false || strpos($_SERVER['REQUEST_URI'], '\plugins\fusioninventory/') !== false){
				return;
			}
		}

I added
		if (class_exists('PluginGlpiinventoryCommunication') && strpos($_SERVER['HTTP_USER_AGENT'], 'GLPI-Agent_') !== false){
			if(strpos($_SERVER['REQUEST_URI'], '/plugins/glpiinventory/') !== false || strpos($_SERVER['REQUEST_URI'], '\plugins\glpiinventory/') !== false){
				return;
			}
		}

After that the enforcement of the plugin can be used again and the agents are able to connect again.
Maybe this can be integrated into the next version?

Added exclusion in latest version https://github.com/DonutsNL/phpsaml

Hi @DonutsNL,
Whit the latest version, in my environment the problem persisted. I fixed it by adding the following line of code in phpsaml/inc/phpsaml.class.php:
private const EXCLUDED_USERAGENTS		= ['FusionInventory-Agent'	=> '/plugins/fusioninventory/',
'FusionInventory-Agent'  => '/marketplace/fusioninventory/',
'FusionInventory-Agent'  => '/plugins/glpiinventory/',
'FusionInventory-Agent_v2.6' => '/plugins/glpiinventory/',
'FusionInventory-Agent'  => '/marketplace/glpiinventory/',
'GLPI-Agent'				=> '/',
'GLPI-Injector'			=> '/'];
I have agents "fusioninventory" still running, pointing to the url glpi.domain/plugins/glpiinventory/
This line of logs/access-glpi.log help me to fix it:
"POST /plugins/glpiinventory/ HTTP/1.1" 302 1065 "-" "FusionInventory-Agent_v2.6"

Hi @loyolajavi, @walkerprog
I can realy use your help with testing.
I added dropdowns in which you can now manually configure the excluded paths. You are able to configure a path,  optionally check the useragent, and configure what saml should do, either enforce auth or bypass it.

Its available in my repo at: https://github.com/DonutsNL/phpsaml
Additional stuff implemented

Add menu button in config menu

Disable login button if no valid config is found

Disable enforce/login if no valid configuration is found
Simplify installation logic hopefully reducing issues with the saml database
implement the GLPI migration logic for future installation and update action moving from 1.3.0 forward.
Addressed all linting issues in the sourcecode

Hi @DonutsNL
I am on vacation right now and will not change anything during my absence for plain reason and really bad internet connection.
Will be back next week and do the update and testing. You should have my feedback by the end of next week.
Sorry for the inconvinience.

Im not in a hurry, happy hollidays  😉😎

Hi @DonutsNL, of course, i will test this tomorrow for a feedback.
Thanks!

That would be great thx 👍🏻💪🏻

Hi @DonutsNL, how are you?
I tried to install the v1.3.0, and it threw an error:

Duplicate entry '1' for key 'PRIMARY'

In the files/_log/sql-errors.log this is the error message:

glpisqllog.ERROR: DBmysql::query() in /var/www/html/glpi/src/DBmysql.php line 379
*** MySQL query error:
SQL: INSERT INTO glpi_plugin_phpsaml_configs
(id,
version,
enforced,
proxied,
strict,
debug,
jit,
saml_sp_certificate,
saml_sp_certificate_key,
saml_sp_nameid_format,
saml_idp_entity_id,
saml_idp_single_sign_on_service,
saml_idp_single_logout_service,
saml_idp_certificate,
requested_authn_context,
requested_authn_context_comparison,
saml_security_nameidencrypted,
saml_security_authnrequestssigned,
saml_security_logoutrequestsigned,
saml_security_logoutresponsesigned,
saml_configuration_name)
VALUES('1',
'1.3.0',
'0',
'0',
'1',
'0',
'0',
'',
'',
'',
'',
'',
'',
'',
'',
'',
'0',
'0',
'0',
'0',
'default');
Error: Duplicate entry '1' for key 'PRIMARY'
Backtrace :
plugins/phpsaml/inc/config.class.php:1279
plugins/phpsaml/hook.php:59                        PluginPhpsamlConfig::install()
src/Plugin.php:915                                 plugin_phpsaml_install()
front/plugin.form.php:51                           Plugin->install()
public/index.php:82                                require()

id 1 already exists in the table "glpi_plugin_phpsaml_configs" it contains the data of v1.2.2.
Or it was not deleted, or the instruction should be an update instead of an insert.

Thanks for the feedback.
I know what the problem is and didnt get to fix it yet. Im implementing the GLPI migration object to manage this for us. I think ill address this issue this evening.

I tested with the new code.
After the update, the table glpi_plugin_phpsaml_configs was not recreated
So, there is no config now:
glpisqllog.ERROR: DBmysql::query() in /var/www/html/glpi/src/DBmysql.php line 379
*** MySQL query error:
SQL: SHOW COLUMNS FROM glpi_plugin_phpsaml_configs
Error: Table 'glpi.glpi_plugin_phpsaml_configs' doesn't exist
Backtrace :
plugins/phpsaml/inc/config.class.php:259
plugins/phpsaml/inc/phpsaml.class.php:473          PluginPhpsamlConfig->getConfig()
plugins/phpsaml/inc/phpsaml.class.php:143          PluginPhpsamlPhpsaml::getSettings()
plugins/phpsaml/inc/phpsaml.class.php:132          PluginPhpsamlPhpsaml::init()
plugins/phpsaml/setup.php:174                      PluginPhpsamlPhpsaml->__construct()
src/Plugin.php:1681                                pluginPhpsamlPostInit()
src/Plugin.php:275                                 Plugin::doHook()
inc/includes.php:95                                Plugin->init()
ajax/notifications_ajax.php:36                     include()
public/index.php:82

Please use a plugin reinstall to recover from any inconsistant state.

Oh ok, with the reinstall the table was created Ok, I will test and tell you how works

Thx 💪🏻😎

Good morning,
It works great! I tested the inventory with both fusion and glpi agents, it works Ok and I didn't have to add any path to the exclude.
Also works Ok the menu button.
On the other hand, the disable login button if no valid config is found, i think is not working.. I deleted the idp id and it didn't disable the login button.
Finally, I think the enforce mode is not working, it redirects infinitely.
In logs I see this:
I wait for a minute since click on login, and in that minute this line repeats many times:

[10/Jan/2024:10:16:00] "POST /plugins/phpsaml/front/acs.php HTTP/1.1" 302 988 "https://URLOfMySSOService"

This message also appears (only once)

Usage of signed integers in primary or foreign keys is discouraged, please use unsigned integers instead in glpi_plugin_phpsaml_configs.id. in /var/www/html/glpi/src/DBmysql.php on line 2138, referer: https://URLGlpi/front/plugin.php

Hi @loyolajavi,
Thanks for testing.

I deleted the idp id and it didn't disable the login button.
Yeah the check was somewhat limited :) thats corrected.

Finally, I think the enforce mode is not working, it redirects infinitely.
I have the same effect. Im diving into that one because somehow a redirect loop is created with the enforced setting. Thanks for being so precise 👍

Hi @DonutsNL
I do think for the redirect-loop a new version of the plugin will be necessasary?
As SAML is very crucial for me I would wait for the new version to test all in once.

I have been debugging and found and corrected the issue. The excludes need more testing as it was clearly not excluding acs.php. As a fix I removed acs.php from the excludes and hardcoded this exclude in the sourcecode because removing it manually from the dropdowns would obviously break the plugin.
I pushed the change to the repo

There was an logic issue in the excludes.php logic. This issue caused excludes to only evaluate 1 exclude from the config and would never itterate. As a result acs.php was never evaluated causing the loop issue. This also showed that acs.php should not be configurable as it could be accidently deleted (causing the loop). I corrected both issues in the latest version inside my repo.

Hi @DonutsNL, how are you today?
I've reinstalled the plugin and now the login button works Ok and the loop redirection does not occur anymore.
I have question.. If you logout from the Glpi logout button, works Ok? I really dont know if Glpi or my Idp service is no working as I expected (It don't came back to the login page)
On the agents' side, I have to add this exclude path, with that, fusion and glpiAgent through /plugins/glpiinventory (I have GlpiInventory plugin installed) works Ok:

And just as a comment, I see this in files/_log/php-errors.log:

[2024-01-11 10:42:01] glpiphplog.WARNING:   *** PHP Warning (2): Undefined array key "valid_id" in .../plugins/phpsaml/inc/phpsaml.class.php at line 365
Backtrace :
plugins/phpsaml/inc/phpsaml.class.php:417          PluginPhpsamlPhpsaml::glpiLogout()
plugins/phpsaml/inc/phpsaml.class.php:198          PluginPhpsamlPhpsaml::sloRequest()
plugins/phpsaml/setup.php:204                      PluginPhpsamlPhpsaml->processUserLogin()
src/Plugin.php:1681                                pluginPhpsamlPostInit()
src/Plugin.php:275                                 Plugin::doHook()
inc/includes.php:95                                Plugin->init()
front/logout.php:42                                include()
public/index.php:82                                require()

If you logout from the Glpi logout button (in enforced mode), works Ok? I really dont know if Glpi or my Idp service is not working as I expected (doesn't return to login page)
The flow would be something like this:
This causes GLPI to flush all session auth stuff and redirect you to the configured idp Service LogOff location also clearing the session stuff from the idp. The idp will redirect you back to glpi (should be the main page because I removed the slo.php) where login is enforced and you are redirected back to the (hopefully reset) idp asking you to login again. If successfull the samlresponse will be redirected back to the acs.php and if correct you are logged in again.
Troubleshooting the loop i have tested this flow multiple times without an issue. It might be problematic if the IDP does not provision an logoff url or the logoff url was misconfigured. This would prob cause an inconsistant state that most prob needs to be solved by resetting the idp session by closing the browser or something.

I have tested it today
At first I reverted all my manual changes
checked settings: Plugin enforce is enabled
SAML-Login is working as expected
Logout redirects to GLPI-startpage and triggers immediate relogin - I think I should deep dive into the idp settings at this point
as expected XML delivery from GLPI-agent was broken again the same second i reverted my changes - as far as good, original state recovered
next update of the plugin to newest version (deleted all files in the plugins folder and uploaded new files)
configuration is completely broken after update, all fields on the settings page are empty, no dropdown is working...
reverted to last functional state to copy the old, functional settings
reinstalled update via uninstall and reinstall, now the settings page is working again, but configuration was not transfered
reconfigured plugin
new menu point ist working fine and I really like it
SAML is now working again
no noticable change in login or logoff behaviour
but inventory via GLPI-agent still NOT working :/
added new exclusion via Dropdowns:
name: Bypass GLPI agent
Agent contains: GLPI-Agent_
Url contains path or file: plugins/glpiinventory/index.php
Bypass SAML auth: Yes
and THERE we go, NOW it is working
login and logoff of user still no change noticeable but glpi agent can now drop the xml with the client data
maybe this could be added in the source code?
additionaly, on the settings page:

it states "No valid Ipd certificate details provided or available" - but certificate is provided and saml is working?? wrong check??
it states "A different version of Phpsaml is marked latest. Version 1.2.1 was found in the repository, you are running 1.3.0" - just for the eye, but could maybe be fixed in future releases?

@DonutsNL
thank you very much for your work, appreciate it
for production use: what exactly is the plan now?
Will the changes be written back to the repo from derricksmith?
will your fork be official?

Im not sure. Working from my dev branch surely is not ideal. @derricksmith has been busy merging. Im not sure when he will be able to merge the latest version.


Add Docker Files, Edit .gitignore

Adds files for Docker. Removes package-lock.json from .gitignore to encourage committing it.

I don't know enough about Docker to review this well.

I don't know enough about Docker to review this well.

@boutell I think this might work but I'm not sure how to test it. I want to make sure APOS_RELEASE_ID is being set/used properly.

@boutell I think this might work but I'm not sure how to test it. I want to make sure APOS_RELEASE_ID is being set/used properly.

Thanks for digging into this.

Thanks for digging into this.

OK, great.
On Mon, Feb 1, 2021 at 1:49 PM Brendon Reyell<EMAIL_ADDRESS>wrote:

@breyell commented on this pull request.
In .dockerignore
https://github.com/apostrophecms/a3-boilerplate/pull/9#discussion_r568059528
:

@@ -0,0 +1,2 @@
+node_modules
+RELEASE_ID

I just ssh'd into a container using this image to confirm both RELEASE_ID
and node_modules are there.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apostrophecms/a3-boilerplate/pull/9#discussion_r568059528,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAH27KXBT4EISWDNY7E3R3S43ZU3ANCNFSM4WDLKJ2A
.

--
THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

@breyell @boutell Let me know if you'd like me to pull this all together!


Move repo switcher next to repo commands

Thinking visually it would make more sense to have the 'Quick repository switch' next to the active repository's shortcuts.

This is no longer relevant since I changed from always showing the org commands to only showing them when you have an organization selected. See version 0.0.3.


Security risks regarding to root privileges

As is discussed in #25, it is not necessary to run the whole program as root. On modern Linux witn Capabilities support, CAP_NET_RAW and CAP_NET_ADMIN are enough for this program to run without root. Root will bring possible security exploits.
Therefore, the following workarounds are hereby proposed:

Package the program properly for mainstream package managers (rpm, deb, etc.) with necessary capabilities granted by default (The AUR package and udp2raw-tunnel in archlinuxcn have already implemented this)
Don't mention the need for root permission (except android / openwrt) in documents
Add a warning to stdout when uid=0
If possible, switch to nobody as soon as possible on start
Do not invoke shell commands inside the program. Instead, make use of netfilter APIs.

Think about it: How many average user will or can use Linux as their daily drive without being able to just setcap the executable?

I dont know.But many clients are running on router/virtual machine(openwrt x86).Almost all server are running on VPS(usually root account is the only account allowed to login).
Those guides aim to demostrate an easy/portable way to run udp2raw at first try.They are just a guide for easy usage，as we can put a warning there，we didnt encourage use to use udp2raw in exactly the same way.

Don't mention the need for root permission (except android / openwrt) in documents

we  can remove the requirment at main page of readme.And tell user in guide, “root is only requirent in this guide，you can run it without root.and its suggest to run without root for security”,and show those word in red color.

Package the program properly with necessary capabilities granted by default

If possible, switch to nobody as soon as possible on start

Those will be done in feature.

Do not invoke shell commands inside the program.

This will be done if its proofed that iptc library  is portable. If sepcific version iptc libary only works on specific kernel,we will stick to iptables command.

Add a warning to stdout when uid=0

I will consider about it. Actually there is a warning if uid!=0.

wangyu:

On many VPS(running a destop linux) ,root is just the default account.We dont need to care too much,that is over-design(of document).Just put a warning in those guides ,it will be okay.

PeterCxy:

Root is the default account on any Linux / Unix distribution, does it mean that everything is running under root and we do not need to care about it because it is overdesign?

An example is nginx. It will even setuid to run itself inside an unprivileged user.

I just meant there are no other users avaliable(allowed to login) on those vps by default.
You misunderstood my words.I didnt mean rootless Xorg/ping itself is overdesign. I meant we can just do as those vps providers，Let users switch to a rootless way by themselves(and we can have a warning in red color).If a warning can solves the problem,complexifying the guide(or creating guide branches for different platforms) is an over-design.

posted by wangyu in #25:

On many VPS(running a destop linux) ,root is just the default account.We dont need to care too much,that is over-design(of document).Just put a warning in those guides ,it will be okay.

posted by PeterCxy in #25:

Root is the default account on any Linux / Unix distribution, does it mean that everything is running under root and we do not need to care about it because it is overdesign?

An example is nginx. It will even setuid to run itself inside an unprivileged user.

I just meant there are no other users avaliable(allowed to login) on those vps by default.
You misunderstood my words.I didnt mean rootless Xorg/ping itself is overdesign. I meant we can just do as those vps providers，Let users switch to a rootless way by themselves(and we can have a warning in red color).If a warning can solves the problem,complexifying the guide(or creating guide branches for different platforms) is an over-design of document.
Saying about complexifying ，I didnt meant the addition step is complicated itseft. Just,it cant succeed on  Openwrt/Android，that will cause confusion.

This will be done if its proofed that iptc library is portable. If sepcific version iptc libary only works on specific kernel,we will stick to iptables command.

It should work as long as the netfilter APIs are consistent. However I'm not sure if it works well with ancient kernels that do not have netfilter. Maybe iptables can be used as a fallback machanism.

I will consider about it. Actually there is a warning if uid!=0.

what about

Running as root can be insecure. It is recommended to run this program as non-root user with cap_net_raw and cap_net_admin

for uid = 0, and

You are executing this program as non-root user. Please make sure cap_net_raw and cap_net_admin are granted or this will not work as expected.

for uid != 0

what about

Running as root can be insecure. It is recommended to run this program as non-root user with cap_net_raw and cap_net_admin

for uid = 0, and

You are executing this program as non-root user. Please make sure cap_net_raw and cap_net_admin are granted or this will not work as expected.
for uid != 0

That will be good.

And tell user in guide, “root is only requirent in this guide，you can run it without root.and its suggest to run without root for security”,and show those word in red color.

When the distribution packages are provided, we can just ask the user to intstall the prebuilt packages in the guide. It is much simpler, and the user will not need to run the binary as root since everything is done within the package. It will be not even needed to mention about the root stuff (or like this: "If you do not want to install these prebuilt packages, you may install this program manually, which will require you to grant CAP_NET_RAW and CAP_NET_ADMIN to the binary manually. This is only for advanced users.")

We can write a guide for Linux distribution with packages avaliable.This guide will work as a paralell guide with the original guide,they will refer to each other at some step.
Before the packages way is proofed stable，i will still suggest to use the original guide.Packages hide details，if it fails it will be hard for user to report issue.

For specific situation,user will be running a system with packages at one side,another system with only binary avaliable on the other side(many user run server on vps,client on router).At the moment I cant guareentee the protocal wont be modified(actually this already happened one time),otherwise it will be a burden for developing.The compatibily of packages and binary will be a issue.
Before this issue has been solved,sticking to the binary guide and download client &server at same side will be a  safer way.
This issue also happens if one side is using a new apt repo,the other side is using an old one.

For specific situation,user will be running a system with packages avaliable at one side,another system with only binary avaliable on the other side(many user run server on vps,client on router).At the moment I cant guareentee the protocal wont be modified(actually this already happened one time),otherwise it will be a burden for developing.The compatibily of packages and binary will be a issue.
Before this issue has been solved,sticking to the binary guide and download client &server from same release will be a  safer way.
This issue also happens if one side is using a new apt repo,the other side is using an old one.

For specific situation,user will be running a system with packages avaliable at one side,another system with only binary avaliable on the other side(many user run server on vps,client on router).At the moment I cant guareentee the protocal wont be modified in furture(actually this already happened one time),otherwise it will be a burden for developing.The compatibily of packages and binary will be a issue.
Before this issue has been solved,sticking to the binary guide and download client &server from same release will be a  safer way.
This issue also happens if one side is using a new apt repo,the other side is using an old one.

Do not invoke shell commands inside the program. Instead, make use of netfilter APIs.

We can always run iptables outsideudp2raw without any of these netfilter/iptc stuff, just using a simple wrapper bash script:
setup-iptables
su nobody /usr/bin/udp2raw-tunnel
clean-iptables

With this wrapper-script, iptable rules always got cleanup even udp2raw crashes or force-killed.

With this wrapper-script, iptable rules always got cleanup even udp2raw crashes or force-killed.

Another good way is to add those iptables manually(or put them into rc.local for convenience) and do not use -a option.In this way,one side wont keep receiving RST after oppsite side exits.I suggest this method for a long-term usage.

CAP_NET_ADMIN is no longer mandatory since SO_SNDBUFFORCE and SO_RCVBUFFORCE are disabled by default after commit 934a65e7bf4ceded52252b709ec1fbbf541e20f4

CAP_NET_ADMIN is no longer mandatory since SO_SNDBUFFORCE and SO_RCVBUFFORCE are disabled by default after commit 934a65e

That is great. I'll update my ArchLinux package details later.

We can always run iptables outside udp2raw without any of these netfilter/iptc stuff, just using a simple wrapper bash script:

This is what I am doing with my AUR package for udp2raw. I run iptables with systemd's ExecStartPre and ExecStopPost to ensure that the iptables rules will be cleaned up even after udp2raw has crashed

Another good way is to add those iptables manually(or put them into rc.local for convenience) and do not use -a option.In this way,one side wont keep receiving RST after oppsite side exits.I suggest this method for a long-term usage.

How about using this method? This might be better because otherwise your host will create tons of RST after you exited udp2raw.

How about using this method? This might be better because otherwise your host will create tons of RST after you exited udp2raw.

I think there is no difference here :)

What I m worrying about is,maybe you will get special care from your ISP after your host sent too many unexpected RST.

Nope you will not. Even if you are not using udp2raw, trying to connect to a closed port will definitely generate tons of RST since it is the way TCP behaves. RST happens everywhere so I don't see any problem removing the iptables rule after exiting.

Nope you will not. Even if you are not using udp2raw, trying to connect to a closed port will definitely generate tons of RST since it is the way TCP behaves. RST happens everywhere so I don't see any problem removing the iptables rule after exiting.

The difference is udp2raw does not use a normal tcp protocol. A normal tcp protocol will close the connection after received RST.
Udp2raw server will keep sending heartbeat even if it has received tons of RST,until session expires.
It doesnt simply close the connection because in this way udp2raw can be used on a host without iptables.
So,if the oppsite is a normal TCP server,your host will only generate a few RST then the connection is closed on oppsite side.If the oppsite is a udp2raw server, your host will create tons of RST.

@wangyu- Yes and that will look like SYN flooding ...... This is really a problem then.


Enable Export button for diversity report

As a admin l want to press the export press so that l can download the diversity report in a spreadsheet.

Some work was done around the diversity report that has introduce a bug that disable the export button on the diversity report.

@HalcyonJAC @JeremykJAC @Darrel you did some work on #1081 and #1100 which has introduce bug to the export button on downloading the diversity report. The export button is currently disable, its means the report can't be downloaded by the team.


Update README.md

Added Introduction to Statistical Learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani http://www-bcf.usc.edu/~gareth/ISL/

Sorry for my late response. It is a great resources! Merged.


[Feature Request] Rounded scroll bar

Context

[X] This issue is not a bug report. (please use a different template for reporting a bug)
[X] This issue is not a duplicate of an existing issue. (please use the search to find existing issues)

Description
Hi
I included the monaco editor in my website. But the website use modern style rounded scrollbars. Like this:
*::-webkit-scrollbar {
    width: 16px;
}

*::-webkit-scrollbar-track {
    border-radius: 8px;
}

*::-webkit-scrollbar-thumb {
    height: 56px;
    border-radius: 8px;
    border: 4px solid transparent;
    background-clip: content-box;
    background-color: #888;
}

*::-webkit-scrollbar-thumb:hover {
    background-color: #555;
}

It does no look good if the whole website has rounded scrollbars, but the editor has thick square ones. Can you please provide an option for rounded scrollbars?

Monaco Editor Playground Link
No response
Monaco Editor Playground Code
No response

I did some very basic poking around on the playground and looks like the scrollbars are hand-rolled and are not native browser scrollbars, so you can most likely use normal CSS to style these (as demonstrated in the demo)
.monaco-editor .monaco-scrollable-element .scrollbar.horizontal, .monaco-editor .decorationsOverviewRuler, .monaco-editor .monaco-scrollable-element .scrollbar.vertical .arrow-background

You will need to figure out what classes to target - but it is all here in the browser for your exploration and customization.

So there are two types of Scrollbars widely used:
The Win 10 Style Block ones often with arrwos and the round ones withour arrwos.
You see examples in your screenshot.
How about the idea to make a switch in the monaco editor wether you like the square ones ore the thin rounded ones? With the rounded scroll thumb I mean. That would make it easier for developers to get the right scrollbar for their needs.

Rounded corners:
.monaco-editor {
border-radius: 10px;
}
.overflow-guard {
border-radius: 10px;
}
Rounded slide-bar:
.slider {
border-radius: 10px;
}

border-radius: 10px;
makes them rounded. One step done. Can you also tell me what options to change, to make them half as thick and to increase the opacity?
Is it possible to let the text end at the scrollbar?


mac builds failing on certificate check on www.apache.org for freemarker

--2019-07-30 04:29:14--  https://www.apache.org/dist/freemarker/engine/2.3.28/binaries/apache-freemarker-2.3.28-bin.tar.gz.asc
Resolving www.apache.org... <IP_ADDRESS>, <IP_ADDRESS>, 2a01:4f9:2a:185f::2
Connecting to www.apache.org|<IP_ADDRESS>|:443... connected.
ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=Sectigo RSA Domain Validation Secure Server CA,O=Sectigo Limited,L=Salford,ST=Greater Manchester,C=GB’:
  Unable to locally verify the issuer's authority.
To connect to www.apache.org insecurely, use `--no-check-certificate'.

Ref https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-mac-x64-openj9/304/console and others

Also got this error on https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-mac-x64-hotspot/330/console
origin	https://github.com/adoptopenjdk/openjdk-jdk8u.git (fetch)
Resetting the git openjdk source repository at /Users/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-mac-x64-hotspot/workspace/build/src in 10 seconds...
Pulling latest changes from git openjdk source repository
Fetching origin
From https://github.com/adoptopenjdk/openjdk-jdk8u
   0a4641b..fd1292e  master     -> origin/master
   144ccd1..416002a  dev        -> origin/dev
HEAD is now at 416002a Merge root at HEAD
Removing build/
Pulling latest tags and getting the latest update version using git fetch -q --tags --depth=1
NOTE: This can take quite some time!  Please be patient
fatal: git fetch-pack: expected shallow list


Calendar API getStart()->dateTime

Hello,
When getting the event start time using getStart()->dateTime the value returned is not always what is expected. The date is returned like this:
2016-03-07T12:00:00-07:00
This looks fine at first but the problem is when there is a timezone set at the event level. The example above is taken from an event that actually has the event timezone set as UTC-5.
Is it possible the dateTime can return the actual time set of the event and not the adjusted time the calendar outputs?
I was not sure if this is an issue with the PHP API or if it is something that would have to be changed on Google's end, any insight is appreciated.
Thanks!

Hello!
The dateTime property will always be an RFC3339 timestamp, according to the docs. You can use PHP's DateTime object from there:
$d = new DateTime($event->dateTime);

var_dump($event->dateTime);
// string(25) "2016-03-07T12:00:00-07:00"
var_dump($d->format('Y-m-d'));
// string(10) "2016-03-07"
var_dump($d->format('U'));
// int(1457377200)

Hey thanks for the response. We did end up figuring out the issue was deep within our code and were able to get it working how we need.


MB-11966 document prime request service item database

MB-11966 for this change
MB-11967 for this change
Summary
Technically only the mto_service_item_customer_contacts table needed to have a trigger added for MB-11966, but I figured I'd add it for the mto_service_item_dimensions anyways since that ticket is coming down the pipe.
[Resolved] It doesn't look like any data is being returned in the move history. The event entry is there, but I'm not seeing any of the values I would expect - they're blank. Looks like only the values that are numbers are having this problem?
^ This was because in pkg/handlers/ghcapi/internal/payloads/model_to_payload.go, removeEscapeJSONtoObject was using a map[string]string for the result. This caused json.Unmarshall to dump any int values that it found.
This article does a pretty great job at explaining what I was running into, in particular the section about Decoding JSON to Maps.
The implications for this is that blank values now get returned as null instead of empty strings.
Setup to Run Your Code

💻 You will need to use two separate terminals to test this locally.
Terminal 1
Start the UI locally.
make office_client_run

Terminal 2
Start the Go server locally.
make server_run

Additional steps

Apply the migration in this PR
Access the office application
Login as a prime simulator
Select a move and create service items: Destination SIT and Shuttle (both types of service item codes), also, remember the move code
Logout and login as an office user
Go to the move code that had service items created for it
Open the network tab or react query devtools and observe that there are now entries for the move history for updates on the mto_service_item_dimensions and mto_service_item_customer_contacts table

Verification Steps for Author
These are to be checked by the author.

[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

Backend

[ ] Code follows the guidelines for Logging
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

Messages

:book:
:link: MB-11966

  Generated by :no_entry_sign: dangerJS against 0800ebbe8d67e026da3209fdbc71964052259179

@YanZ777 This is something I noticed before that we aren't doing any error checks when unmarshaling the JSON.  It sounds like you weren't getting an error here, it was just silently dropping non-string types.  Do you mind at least logging the error from the unmarshal call, if not bubbling up any possible errors here?
https://github.com/transcom/mymove/pull/8808/files#diff-2aaf788a6f0f8978cc3c4b19155cd99a084d553a2d49c70add1fcba5e6dcaf60R291

@duncan-truss I ended passing appCtx to the payloads.MoveHistory function. Is that what you had in mind?

@duncan-truss Thanks for reviewing, and good catch! I ended passing appCtx to the payloads.MoveHistory function. Is that what you had in mind?

Yes that's exactly what I was suggesting.  The other alternative would be to have the payload methods return the error all the way back to the handler along with the payload and do an error check there if you didn't want to pass the appCtx around everywhere.  Luckily the move audit history isn't used everywhere like some other payload methods that would require updating a bunch of files.  I think that's a big reason we don't have error responses in our payload methods but probably should since they can fail.

I don't know how you all are testing these audit logs changes when adding something new just on the backend side or do you wait and then test with the handler?
I tested with the prime simulator creating a destination SIT service item and a crating service item and the query seemed to capture them in the move history when I looked in the network tab.

For checking that the trigger is working and that things get written:
I've been using the prime simulator to create both those types of service items and then checking the db to see that there's stuff captured in the audit_history table. Using postman should also work too.
For checking that the fetcher is working:
You can go to the corresponding move history page (logged in as a TIO or TOO) for the move you added those service items to, and then check the network tab and see what the payload returns. You can also use the React query debugger to check as well.

I don't know how you all are testing these audit logs changes when adding something new just on the backend side or do you wait and then test with the handler?
I tested with the prime simulator creating a destination SIT service item and a crating service item and the query seemed to capture them in the move history when I looked in the network tab.

For checking that the trigger is working and that things get written: I've been using the prime simulator to create both those types of service items and then checking the db to see that there's stuff captured in the audit_history table. Using postman should also work too.
For checking that the fetcher is working: You can go to the corresponding move history page (logged in as a TIO or TOO) for the move you added those service items to, and then check the network tab and see what the payload returns. You can also use the React query debugger to check as well.

I was thinking more how you all approach automated server tests for each of the tables since it relies on the handlers setting different event names and context.

I was thinking more how you all approach automated server tests for each of the tables since it relies on the handlers setting different event names and context.

Oh gosh, I'm not even sure how we would go about having automated server tests for each of the tables if you're talking about testing the triggers.
For the most part, I think we've been testing the move history fetcher and making sure that it's retrieving the data we expect. It's kind of more of an integration test rather than a unit test at that point I think since it tests the trigger and the retrieval.
That did remind me that I needed to add tests for those two tables I added, which I've done.


[HUDI-2410] Fix getDefaultBootstrapIndexClass logical error

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

What is the purpose of the pull request
public static String getDefaultBootstrapIndexClass(Properties props) {
String defaultClass = BOOTSTRAP_INDEX_CLASS_NAME.defaultValue();
if ("false".equalsIgnoreCase(props.getProperty(BOOTSTRAP_INDEX_ENABLE.key())))
{ defaultClass = NO_OP_BOOTSTRAP_INDEX_CLASS; }
return defaultClass;
}
When hoodie.bootstrap.index.enable is not passed, the original logic will follow HFileBootstrapIndex，
This should not be judged here
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@hudi-bot run azure

@hudi-bot run azure

@SteNicholas Please help to review.


Make path lookup to be case sensitive

Fixes https://github.com/gollum/gollum/issues/824

Looks good to me! Thanks @0sc


Refactor parser unit tests and fix test failures.

All unit tests now run and pass.
This pull request also has a few UT-execution enhancements:

Instead of explicitly including all tests runtests.jl now automatically runs all "*.jl" in the test directory.
You can now iterate on one or more unit tests by pushing a test name into ARGS while in the REPL.

push!(ARGS, "parse_psse")
include("test/runtests.jl")

You can customize logging verbosity by setting the environment variable PS_LOG_LEVEL to Debug, Info, Warn, or Error.

You will also notice a different in unit test output.  When all tests pass it will show the total number of tests in PowerSystems.  If any test fails then it will print out a summary table showing the results of each test.  The difference is because of a top-level @testset statement that I added in runtests.jl.

Codecov Report

Merging #154 into master will increase coverage by 9.62%.
The diff coverage is 61.11%.

@@            Coverage Diff             @@
##           master     #154      +/-   ##
==========================================
+ Coverage   47.16%   56.79%   +9.62%
==========================================
  Files          43       44       +1
  Lines        3763     3768       +5
==========================================
+ Hits         1775     2140     +365
+ Misses       1988     1628     -360

Impacted Files
Coverage Δ

src/parsers/dict_to_struct.jl
88.32% <ø> (+0.5%)
:arrow_up:

src/PowerSystems.jl
100% <ø> (ø)
:arrow_up:

src/parsers/pm_io/multiconductor.jl
0% <0%> (ø)
:arrow_up:

src/parsers/im_io/data.jl
0.58% <0%> (ø)
:arrow_up:

src/parsers/im_io/common.jl
92% <100%> (ø)
:arrow_up:

src/common.jl
100% <100%> (ø)

src/models/branches/lines.jl
100% <100%> (+42.85%)
:arrow_up:

src/parsers/im_io/matlab.jl
88.46% <28.57%> (+3.2%)
:arrow_up:

src/parsers/pm_io/data.jl
44.38% <33.33%> (+4.71%)
:arrow_up:

src/utils/data.jl
33.33% <37.5%> (-36.67%)
:arrow_down:

... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 713591a...f7b5427. Read the comment docs.

Can you rebase and merge?

Done.


Hang vs flow vs third thing

Hey Kim!
One of the things I do in a project that doesn't use zprint is format medium/long function calls on two lines instead of either hanging or flowing them.
Here's a lightly changed example from the zprint-enforced codebase (indentation is intentional):
;; desired
      (let [example (data.example/get-by-org-id-and-items
                      db-conn true [org-id] {:very-long-arg-here true})]
;; vs hang (how zprint formats it at the moment)
      (let [example (data.example/get-by-org-id-and-items db-conn
                                                          true
                                                          [org-id]
                                                          {:very-long-arg-here
                                                           true})]
;; vs flow (better than hang but zprint refuses to do it, idk why or how)
      (let [example (data.example/get-by-org-id-and-items
                      db-conn
                      true
                      [org-id]
                      {:very-long-arg-here true})]

Is this possible to do with zprint?
Related, is it possible to tell zprint to not hang a function call if the function symbol is too long? Having lots of indentation already sucks but then to have 36 additional characters of empty space really harms readability in my opinion. Maybe it's due to zprint seeming to vastly prefers to hang vs flow. I personally don't like hanging except for rare occasions/specific symbols, so it's hard to know what's intentional vs what's just a by-product of how my company writes code lol.
Thanks for all the help and quick feedback.

An interesting question, thanks!  I can get zprint to do what you want in some cases, but when space is tight it might not end up being what you want.  An example:
(czprint i261 {:parse-string? true :fn-map {"get-by-org-id-and-items" [:guided {:guide [:element :newline :element-best-*]}]}})
(let [example (data.example/get-by-org-id-and-items
                db-conn true [org-id] {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

That's pretty much what you want, I think.  But when space is tight, this is what you will get:
[I've narrowed the :width to simulate this expression being closer to the right margin, not because I expect you would ever use a :width that narrow.]
; :width 60

(czprint i261 {:parse-string? true :fn-map {"get-by-org-id-and-items" [:guided {:guide [:element :newline :element-best-*]}]} :width 60})
(let [example (data.example/get-by-org-id-and-items
                db-conn true [org-id] {:very-long-arg-here
                                         true})]
  (some-body expressions)
  (more-body expressions))

; :width 55

(czprint i261 {:parse-string? true :fn-map {"get-by-org-id-and-items" [:guided {:guide [:element :newline :element-best-*]}]} :width 55})
(let [example (data.example/get-by-org-id-and-items
                db-conn true [org-id]
                {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

I'm guessing that is not what you want.  I'm thinking that you want the second line to format indented under the first line (as in the first example) if the second line fits within the width, but if it doesn't fit, I'm guessing that you want just a flow:
(czprint i261 {:parse-string? true :width 55})
(let [example (data.example/get-by-org-id-and-items
                db-conn
                true
                [org-id]
                {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

I can think of several ways to implement what I think you want.  I could extend the guide capability to allow it, I could imagine a new fn-type which would just do it explicitly.
It sounds like you have enough big functions, that having some way to configure this sort of formatting based on function size would be useful -- so that you don't have to put each big function in the :fn-map.
I'll look into how to do both of those things.  Seems like a lot more fun than rewriting one of the complex release tests, which I'm in the middle of just now.
Your last point was, I think, if there is any way to get zprint to prefer hangs less than it does now.  There are a lot of heuristics that drives the formatting to be something that looks good -- to me, at least.  These are all changeable.
That said, the interactions between them are complex.  Some of the more simple examples:
 (czprint i261 {:parse-string? true :tuning {:hang-if-equal-flow? false} :list {:hang-size 4}})
(let [example (data.example/get-by-org-id-and-items
                db-conn
                true
                [org-id]
                {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

The one above says that if the number of things in the hang is > 4, don't hang it.
(czprint i261 {:parse-string? true :tuning {:hang-if-equal-flow? false :hang-flow 0.7}})
(let [example
        (data.example/get-by-org-id-and-items
          db-conn
          true
          [org-id]
          {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

This one just turns down the intent to hang things.  You might notice that it also didn't hang the binding for example, so that may not be what you want.
I've demonstrated these two tunings by changing the configuration for everything.  You could just change the tunings inside of specific functions to get those functions behave differently if you wanted.    You can turn them off for all lists (which are function calls, but not binding pairs) like this:
(czprint i261 {:parse-string? true :list {:hang? false}})
(let [example (data.example/get-by-org-id-and-items
                db-conn
                true
                [org-id]
                {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

This will affect function calls for functions that are not already in the :fn-map.  If the function is in the fn-map, then this shouldn't affect it.  That might be what you want, but I kind of doubt it.
You might want to try the three things above (or with slightly different numbers) on your code base to see if it generates things that you like better.  If so, great.  If not, I'd be happy to work with you to find other tunings (of which there are probably 7 or 8 more), each of which interacts with some of the others, to find something that is more to your liking.
I'll work in any case on handling the long function name situation at the top of this issue.

Wow, what a helpful and detailed response. Love asking you questions.

It sounds like you have enough big functions, that having some way to configure this sort of formatting based on function size would be useful -- so that you don't have to put each big function in the :fn-map.

If by "big" you mean "many characters", then exactly right. We currently have roughly 4k defns in our codebase and can't possibly list all of them.
:guided is cool. I don't see it in the reference, is it detailed somewhere else? You're right it's not exactly what I want but it's close and something built off of it would be great.
I suspect for my hang preference, I'm just looking to tweak the numbers a little to make it less aggressive, as you suggested. I'll play around with it and see what I can do!
Thank you for all the help.

The whole "guide" capability is something that I built originally intending to document for general use, but it is so complex that it would take a many pages (and more than many days) to fully document it.  Now, that wouldn't be terrible, since my current working notes aren't all that clear. Since I did it several years ago, I'm sometimes wondering what this or that construct does.  That said, the guide I put in the above example would work in general to give you what I demonstrated there.  There isn't a way to use the current guide capability to get closer to what you want, as you can't say "if this whole thing fits on a line do this, or otherwise do that", either in a guide or in an :option-fn which would generate a specific guide for just a single situation.
I'm still thinking about where would be best to actually implement what you want.  It will be at least a week before I even start on it, as I'm tied up right now, so I'm mulling it over.

There is truly no rush here, our code won't be hurt if it keeps looking how it's looked for the past 5 years. Thanks for all the hard work you do.

I've think I've got something workable here for you. It won't ever hang it, and tries to put the remainder of the function expression on the next line, but if it doesn't fit, it flows it.  Some examples:
; 80 columns

(czprint i261 {:parse-string? true :fn-map {"get-by-org-id-and-items" [:guided {:guide [:element :newline :element-wrap-flow-*]}]} :width 80})
(let [example (data.example/get-by-org-id-and-items
                db-conn true [org-id] {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

; 60 columns

(czprint i261 {:parse-string? true :fn-map {"get-by-org-id-and-items" [:guided {:guide [:element :newline :element-wrap-flow-*]}]} :width 60})
(let [example
        (data.example/get-by-org-id-and-items
          db-conn true [org-id] {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

; 55 columns

 (czprint i261 {:parse-string? true :fn-map {"get-by-org-id-and-items" [:guided {:guide [:element :newline :element-wrap-flow-*]}]} :width 55})
(let [example (data.example/get-by-org-id-and-items
                db-conn
                true
                [org-id]
                {:very-long-arg-here true})]
  (some-body expressions)
  (more-body expressions))

Three different widths, with different results.  How does it look to you?
I haven't started on the "configure it this way if the size of the function name is > some specified number".  This is just what would happen if the function name was longer than the specified number.

These look good! I think it's a nice option and if I can say "use this format if symbol is longer than 20 characters", I'll put it to work immediately.

Great!  Just to be clear, the above examples require new code in the "guide" capability that will only be available when 1.2.5 is released.  Which should not be that long, I'm thinking in a couple of weeks or so.

Gosh I wish you drove all the OSS projects I used, @kkinnear. Always learn something and feel very oddly attended-to when you I read these FRs.
(Not just here to say the above, I also would really love a max-symbol-length-hang configuration)

Thanks!  I appreciate the good words, it makes a difference.
I'm working on the underlying capabilities for the max-symbol-length detection.  It is taking longer than expected, unfortunately, as I had to re-architect the implementation of the basic :option-fn implementation.  But I'm still hoping to release it soon-ish.

Your third thing -- handling long function names differently, is now implemented in 1.2.5.  You have to configure a built-in :option-fn to get what you want, but there is an example which does exactly what you asked for.
Here is the explanation from the CHANGELOG.md:
  * A couple of new capabilities.  Several people have wanted regular
  expression capabilities in the `:fn-map`.  A couple of people have also
  wanted a way to format functions where the function name was over
  some length differently than they are usually formatted.  Both capabilities
  are now available by using a built-in `:option-fn`.  See `:rule-example`
  (and `:regex-example`) in the `:style-map` for some worked examples.  The
  `:option-fn` `rulesfn` accepts a vector of pairs of elements as its
  first argument.  If the left-hand-side of a pair is a function, it will
  call that function with the string format of the function name. If the
  function returns non-nil, then the right-hand-side of the pair is returned
  as the option map.  If the left-hand-side of the pair is not a function,
  it is assumed to be a regular expression, and it is matched against the
  string format of the function name.  If it matches, then the right-hand-side
  of the pair is returned as the option map.  The pairs in the vector are
  processed in order until one of them has the right-hand-side returned
  as an option map or they are all completed, in which case the `:option-fn`
  returns nil.


Port information reversed to what standard docker tools display

Hi, thanks for nice work!
One thing, I noticed that ctop displays port something like this:

whereas standard docker tools reverse inner/outer ports mapping info:

This is confusing.

Good point @dodalovic -- this has been reversed in https://github.com/bcicen/ctop/commit/58d9e4e194a010a9b8cb2eb3537987f9d87497bd


Forms: Common base state

Summary of changes
Introduced a new FormElementState which is an abstract class that forms the state object for a FormElement type. Since in the SDK all the element types inherit from a FormElement this state class acts as a base class for any other derived FormElement state objects. This then creates the following inheritance heirarchy

FormElementState

BaseFieldState

FormTextField
.. other field element type states

BaseGroupState
RelationshipElement state (future)
AttachmentElement state (future)

thanks @sorenoid, merging!


Login failed from frontend with "CSRF token not found in request"

While trying to login:

Config file

@Goxiaoy can you please check this also?

Fixed.
Notice: this project is not production ready. use at your own risk


feat: enable DEX button

https://github.com/aeternity/superhero-wallet/blob/28778513e0d3b1f9882780bcdd3ac0500a0df13a/src/popup/router/pages/FungibleTokens/TokenDetails.vue#L174
Shall we maybe also change the wording?


When registering formatters, also register a new command

Currently the best practice for creating formatter extensions is to register them with registerDocumentFormattingEditProvider. This is great because it makes everything work together and extension authors don't have to create their own formatting implementation, just the logic behind it. However, the user experience suffers somewhat because if they have several formatters, they can't choose to run just one of them on a document.
For example, I have two formatter extensions for Python. One of them is yapf, which formats the whole document but does not affect docstrings. Another of them is a custom extension that formats docstrings. I'd sometimes like to run just one of those and not the other, but that's not possible unless each extension also registers a second command which defeats the advantage of the Formatter API.
Instead, when registering a formatter using the API, VSCode should simultaneously register a command like Format Document with {X}.

fun timing: https://github.com/Microsoft/vscode/issues/41882#issuecomment-475281706


Linked line color/styles with React

I'm having trouble setting a non-default color for the linked lines. Trying to follow the README instructions:
.linkUpdate((d, i, arr) => {
  d3.select(this).attr('stroke', 'red');
  // d3.select(this).attr('stroke-width', 2);
})

Here's a forked js sandbox.
https://stackblitz.com/edit/d3-org-chart-react-integration-hooks-shjp76?file=OrgChart.js
Getting an error on run/build:
this.setAttribute is not a function

Is there another way to accomplish this with the react instantiation?

You are using the arrow function and this does not refer to the correct object. Pass a function callback instead


Allow xaxis column to be set. Allow suppression of invalid data columns

Hi,
When somebody brought this plugin to my attention I immediately fell in live with it and simple had to write a Drupal module for it: http://drupal.org/project/highcharttable
When playing with it I felt two extensions would be really useful. They're both included with this single commit.

Allow the data-graph-xaxis="#" attribute to be specified on the table to specify as the horizontal axis a column other than the first, i.e. 0.

I found that Highchart does not cope very well when one of the series contains invalid data, like for instance a column of words, rather than numbers. Not only does that column not display, none of the valid numeric columns display either. The chart is blank. To avoid this this commit introduces datagraph-suppress-invalid-series="1" so that the chart will correctly display even when some of the columns contain invalid data.

I've marked my changes "RdB", but please feel free to remove those code comments.
I hope you agree with these changes and can create a new release soon.
Thanks again for all your work on this plugin.
Rik

Did you have an example of use case where the data has to be cleaned by highchartTable and not before, when generating the table ?
For the data-graph-xaxis attribute, could you open a separate pull request to discuss that with exemples on jsfiddle ?

@RikdeBoer any news on this PR ?


HMR runtime fails to run dependency accept handlers when a non-handled child is in the same update as an accepted one

Possibly a bug with Harmony modules, but I am not sure.
getAffectedStuff in HotModuleReplacement.runtime.js calls delete outdatedDependencies[parentId] every time a dependency is not in the parent's parent.hot._acceptedDependencies, which might happen when a module is directly (or indirectly) required by multiple parents and any of the parents doesn't have an accept handler.
Removing it from outdatedDependencies then prevents the accept handlers from running.

The screenshot provided in #2649 happens to show one case that is affected by this.
It looks like the situation is hairier and kinda opposite than what I actually described above.
getAffectedStuff returns modules [175, 299, 287, 174, 118, 300, 279, 805] (the first time) in that example. Module 175 is the file that was edited.
Module 175 is required by 299 and 287. 287 is required by 174. 174 is required by 118. 118 re-exports one of the symbols imported from 174.  299 is required by 300. 279 is the entry point, and it requires both 118 (including the reexported symbol) and 300. 805 is webpack's multi loader. ASCII art representation:
    /---- 299 - 300 ----\
175 -- 287 - 174 - 118 -- 297 - 805

Module 279 requires 118 and registers an accept handler for it. getAffectedStuff first processes 118, sees its parent 279 has an accept handler, and sets outdatedDependencies[279] = [118]. When getAffectedStuff then gets around to 300, it sees that it also has parent 279, and proceeds to delete outdatedDependencies[279], preventing the accept handler from running and thus ruining everything.

The accept handler in a module is only called when the module is not disposed, because a module don't need to update it's dependency if it will be disposed anyway.
If there would be no connection between 299 and 300 the accept handler in 297 would be called and 297 would not be disposed. But there is this (unaccepted) way 175 -> 299 -> 300 -> 297 which means 297 must be disposed and accept handlers are not called.
Normally when an update bubbles up to the entry point the HMR fails and the page is reloaded. But this doesn't happen here (which is a bug). I assume you are using the CommonsChunkPlugin?

Yes, I am using CommonsChunkPlugin.
What I am trying to do here is hot-reloading redux reducers. 175 defines a reducer and it's action creators, 174 is the root reducer, 118 is a store factory. 297 sets up a store with the factory, and (tries to) accept changes to the store factory. As the store factory reexports the root reducer, this also depends on the root reducer (it would've been directly imported anyway otherwise).
299 is a component that happens to use 175's action creators, so it also appears in the chain. They're both Harmony modules, so in theory if 175 gets updated, the binding that 299 sees should be updated as well, even without a specific accept in here.
It looks like I can work around the issue by splitting the reducer and action creator files (and not adding a wrapper index.js that reexports them), but this makes the "user" API less ergonomic...

so in theory if 175 gets updated, the binding that 299 sees should be updated as well, even without a specific accept in here.

You still need to accept it. But you can omit the accept handler in this case: module.hot.accept(175)

I see. So the problem is that the update is not outright rejected by the runtime for not having accepters in all paths.
This does look like it'd be hard to debug in some cases, though ("why is this module not hot reloading, I have accept handlers everywhere (*but one place I don't realise)"), but I guess fully refusing it is better than incorrectly accepting it.

@sokra Is only-dev-server supposed to give { ignoreUnaccepted: true } to apply? This prevents it from rejecting the update (and explaining why). only-dev-server is then just supposed to not reload, but it still should output diagnostics.

This does look like it'd be hard to debug in some cases, though ("why is this module not hot reloading, I have accept handlers everywhere (*but one place I don't realise)"), but I guess fully refusing it is better than incorrectly accepting it.

We could add some more information to the message, including the path from the updated module to the root.

The HMR runtime was improved. Error message was improved.


Adobe Experience platform MSSQL connector not working

MS SQL connector has a specified connection string format which doesnt seem to be working and there is not even a sample string provided in the documentation.
Can someone please help if they have setup this connection in the past?

@shrivabee can you please provide the link to the documentation that you are using?

Here it is : https://experienceleague.adobe.com/en/docs/experience-platform/sources/ui-tutorials/create/databases/sql-server

hey @shrivabee let me forward your questions to engineering and get back to you asap. thanks!

@shrivabee I have heard from sources engineering and they suggested that you file a support ticket as this may require some engineering troubleshooting. Also, do you have access to the internal slack? If so, please post your question about SQL native vs windows authentication in the #aep-sources-users channel for a quicker response. In the meantime, I will update the connection string examples

Closing this issue. Descriptions and examples for MS SQL Server source have been updated.
API: https://experienceleague.adobe.com/en/docs/experience-platform/sources/api-tutorials/create/databases/sql-server#gather-required-credentials
UI: https://experienceleague.adobe.com/en/docs/experience-platform/sources/ui-tutorials/create/databases/sql-server


Malt DB fails because of Python "distutils" module depricated

Currently, MALT DB building step fails with the following error:
File "/home/nikolay/miniconda3/envs/aMeta/lib/python3.10/site-packages/snakemake/deployment/singularity.py", line 11, in <module>
    from distutils.version import LooseVersion
ModuleNotFoundError: No module named 'distutils'

This seems to be related to the evolution of python and the fact that malt-build.py is driving the database building. We previously had another problem with malt-build.py because of slight changes in python syntax. I would suggest to re-write this rule via plain shell instead of python.

Fixed in the latest PR


Support Large v2

https://github.com/openai/whisper/commit/4179ed2475cc84cba66868b516232ef1b74dacdf
OpenAI seems to have shipped a new version of the large model.

The "large-v2" model is trained for more epochs with regularization and shows improved performance compared to the previous large.
It has the same architecture as the original large model.
When load_model("large") is called, the "large-v2" model will be loaded.
We will soon update the paper regarding this new model.

just pushed a version supporting this :)

Hi @chenxwh ,
Thanks for all your great work on this!
Just wanted to let you know that this new change is causing failures for models requesting the original "large" model that are set to use the latest model version.
Failing:
model = replicate.models.get("openai/whisper")
prediction = replicate.predictions.create(
        input={
                "audio": signed_audio_storage_link,
                "model": "large",
                "translate":False
            },
            version=model.versions.list()[0] # select latest model version
            )

Fix:
model = replicate.models.get("openai/whisper")
prediction = replicate.predictions.create(
        input={
                "audio": signed_audio_storage_link,
                "model": "large-v2",
                "translate":False
            },
            version=model.versions.list()[0] # select latest model version
            )

yes now it is set to use large-v1 for the original and large-v2 for the new weights

Ah, I see.  I was using the above code in production and started getting errors, just left this here to help! I suppose I should pin the version number instead of always grabbing latest.
Cheers and thanks again!

Will https://replicate.com/openai/whisper be updated with this?


Feedback from Dave's PR #6

Redo folder structure so src/ matches Craftwork\IdObfuscator\
Add a built in hex character set
Use self keyword instead of class name for return types
Improved InvalidCharacterSetException message
Add tests to ensure character sets with extended ASCII fail
Use null coalesce operator when instantiating the fallback CharacterSet in IdObfuscator
Cache the result of the length method in the CharacterSet

I merged Dave's PR in to a branch (replaced-phpspec-with-phpunit) so that master isn't in any sort of unfinished state. Then I forked the project, did the feedback that we left on Dave's PR, merged the changes in to master on my own repo, and now this is the master > master PR.
Unfortunately I didn't really think it through well because now this PR is both Dave and my changes combined.


New devices from Teckin not working

Good morning,
I'm going to explain my problem to see if someone can help me understand the use of the plugin and the things that happen to me. I will leave everything very well documented so that the failures can be understood.
I have the Homebridge installed on a Raspberry Pi 4 with 4Gb ram that has the latest Raspbian OS system. At home all the products I have are from the Teckin brand as an example:
Smart plug model: SP22 (measures watts, amps, etc)
Simple plug model: SP21 (only On / Off)
Led strips model: SL07 (RGB, On / Off, Brightness, etc)
I have managed through the video to get all the IDs and Keys of the devices, but at the moment that I adopt them in the Homebridge I cannot interact with them. I press the Homebridge button but the accessory does nothing.
I attach the configuration that I have in the Homebridge:
{
"platform": "TuyaLan",
"name": "TuyaLan",
"devices": [
{
"name": "Enchufe Watt",
"type": "Outlet",
"manufacturer": "TECKIN",
"model": "SP22",
"id": "bf051c0516bd4ba7f30xxx",
"key": "69262abe946edxxx",
"ip": "<IP_ADDRESS>",
"voltsId": 20,
"ampsId": 18,
"wattsId": 19
}
]
}
With this configuration in the Homebridge terminal this appears:
[TuyaLan] Starting discovery...
[TuyaDiscovery] Discovery started on port 6666.
[TuyaDiscovery] Discovery started on port 6667.
[1/14/2021, 6:42:49 PM] Homebridge is running on port 51564.
[1/14/2021, 6:42:50 PM] [TuyaLan] Discovered a device that has not been configured yet (bf51557fe924d09c2bjkkd@<IP_ADDRESS>).
[1/14/2021, 6:42:51 PM] [TuyaLan] Discovered a device that has not been configured yet (bf90468666bbd51871loib@<IP_ADDRESS>).
[1/14/2021, 6:42:52 PM] [TuyaLan] Discovered a device that has not been configured yet (bffe9a3019604754e2jy3r@<IP_ADDRESS>).
[1/14/2021, 6:42:54 PM] [TuyaLan] Discovered a device that has not been configured yet (bfda5e6553f02c1f80fw0j@<IP_ADDRESS>).
[1/14/2021, 6:42:58 PM] [TuyaLan] Discovered a device that has not been configured yet (bf08a130a876a55b6ejiog@<IP_ADDRESS>).
[1/14/2021, 6:42:58 PM] [TuyaLan] Discovered a device that has not been configured yet (28201518d8bfc0d93051@<IP_ADDRESS>).
[1/14/2021, 6:43:00 PM] [TuyaLan] Discovered a device that has not been configured yet (bf7d59b70f25376103rtqp@<IP_ADDRESS>).
[1/14/2021, 6:43:49 PM] [TuyaLan] Failed to discover Enchufe Watt (bf051c0516bd4ba7f30dce) in time but will connect via <IP_ADDRESS>.
[1/14/2021, 6:43:49 PM] [TuyaLan] Connected to Enchufe Watt
[TuyaAccessory] Sending first query to Enchufe Watt (3.1)
[TuyaAccessory] Malformed update from Enchufe Watt with command 10: ��}=��l�[_����=�&���0a�
[TuyaAccessory] Raw update from Enchufe Watt (3.1) with command 10: 000055aa000000000000000a0000002c00000001b31b1d2c5ffd7d153d99ed876c8a1a5b5ffbb2e9b5d23dcb1f26a9ade83061f521f335050000aa55
But in the list of Homebridge accessories I see a device that is my plug but pressing it turns the button on and off but does not transmit the command to the plug.
I would like to know what I am doing wrong and understand how I have to do it to make it work. The devices have been adopted in the Smart Life application but I have the App completely closed when I do the tests.
Can someone please tell me what is going on?
Thank you very much and a good day.

I'm getting the same logs when I try to connect to my Carro ceiling fan.
Config:
"platform": "TuyaLan",
"devices": [
{
"type": "Fanlight",
"name": "Ceiling Fan",
"id": my correct id here,
"key": my correct key here,
"ip": "my correct ip here",
"manufacturer": "Carro",
"model": "Smart WiFi Fan Switch",
"useBrightness": "True",
"maxSpeed": 10
}
]
Logs:
[11/29/2023, 11:08:49 AM] [TuyaLan] Failed to discover Ceiling Fan (my correct id here) in time but will connect via my correct ip here.
[11/29/2023, 11:08:49 AM] [TuyaLan] Connected to Ceiling Fan
[Tuya] Malformed update from Ceiling Fan with command 10: E5�s%#��r'���{�V����  A�
[Tuya] Raw update from Ceiling Fan (3.1) with command 10: 000055aa000000000000000a0000002c000000014535901d1673251023059daf722702d202d102b47bd9561be1fbeed7f60941ce9a6783610000aa55

Same here with my ceiling fan.
[8/5/2024, 11:42:56 PM] [homebridge-tuya] Connected to fan
[8/5/2024, 11:42:56 PM] [homebridge-tuya] Malformed update from fan with command 10: �jTEGfz�1Ig{��x�8�sim��:��<�v@{
[8/5/2024, 11:42:56 PM] [homebridge-tuya] Raw update from fan (3.1) with command 10: 000055aa000000000000000a0000002c00000001cd6a544547667ac83149677bb7c578a30538c873696dfc9f3a83a93cb476407bb33a18700000aa55


feat: mu support

just tested this, and it works! :)

Any chance for a readme update?

@knownasilya, yeah, nice catch! I'm planning to add more tests and then write down readme update

@lifeart thanks for all the work here! Just wanted to give you heads up that we've got a big deadline early next week, so it's likely we won't have a chance to review this until then. But don't take the silence as lack of interest!

Hey there @davewasmer - wondering if this PR is still in play? Would love to try this out in Octane :)

@lifeart I thing the last thing holding this up is our hesitance with the MU part, since that's an ended experiment.

Using https://github.com/lifeart/ember-language-server/issues/12 it's possible to implement ELS support for this feature, for example:
find text content in template
https://astexplorer.net/#/gist/97bd072e0190bc366b6e78d0801df678/84c54a1f052c2924c29a828773a9657d1dfb8374
parse it using babel parser
https://astexplorer.net/#/gist/4d939c5fe32758eaf10155e2c9140353/cdf596d302dd1e0aba625a41588dccdd7f14181b
resolve component name and return it back

:tada: This PR is included in version 1.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Use IndexMap for graph_greedy_color output

Right now functions that use or wrap a HashMap for their return type to
Python have a non-deterministic iteration order. This is because HashMap
is by definition an unordered type. However, the expectation for Python
users is that since Python 3.6 dictionaries are ordered as they preserve
insertion order. To fix this user expectation for graph_greedy_color()
this commit leverages IndexMap from the indexmap crate [1] which offers
similar performance but offers consistent order (and slightly faster
iteration, although for our use case it might not matter). This will
provide a consistent insertion order for the output dict from
graph_greedy_color().
Wider use of IndexMap for other places we return or wrap HashMaps to Python
is being done in #386 to get this advantage more broadly, but it depends on
a new PyO3 release to work. So in the meantime this just fixes the issue
for graph_greedy_color() where this a reported issue and need for the
deterministic iteration order (it's also doesn't require the PyO3 side
changeseasy as it returns a PyDict instead of relying on the pyfunction
macro to build to convert for us).
Fixes #347
Co-Authored-By: Ivan Carvalho<EMAIL_ADDRESS>[1] https://crates.io/crates/indexmap

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.01%) to 96.209%

Files with Coverage Reduction
New Missed Lines
%

src/lib.rs
1
97.72%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
8425

Relevant Lines:
8757

💛  - Coveralls


Question: can lifecycled process requests for other instances?

One use case for lifecycled is that you run it on all EC2 instances, and when an instance terminates, you run the action on the instance that is terminating.
Another use case is to run those actions on a single/central management instance, rather than the instance that is terminating. There might be a variety of reasons for this, such as reliability, management access or permissions constraints in the env.
With a review of the go code, it seems lifecycled is restricting actions to the instance it is running on, is that true?

#79 is somewhat related to this issue

With a review of the go code, it seems lifecycled is restricting actions to the instance it is running on, is that true?

Yep, each instance will only act on termination events that are targeting its own instance ID. The difficulty with running it centrally is that you'd need another way to initiate the shutdown script on the host that is being terminated, which I believe would increase complexity a lot 🤔

The difficulty with running it centrally is that you'd need another way to initiate the shutdown script on the host that is being terminated, which I believe would increase complexity a lot 🤔

For some use cases, the actions taken are with a central system and can be run from a single instance responsible for running a tool like lifecycled.

@itsdalmo / @lox, would making the filter optional be enough to make this a reality?


License?

Great library, thinking about using this, but before I do could you add a LICENSE to this repo? Just want to make sure you are giving explicit permissions.  Relevant

Sure, I added a license/unlicense file.


trained models' link

the link of checkpoint of the trained models is unavailable

@MorchelPeng Thank you for finding this issue. I deleted some antique files a few months ago and this checkpoint might be included. I would try to resume the file.