Build Error - VPN Server module

Error: External Program Execution Failed
│
│   with module.vpn_module.module.clis.data.external.setup-binaries,
│   on .terraform/modules/vpn_module.clis/main.tf line 22, in data "external" "setup-binaries":
│   22:   program = ["bash", "${path.module}/scripts/setup-binaries.sh"]
│
More details

https://github.com/terraform-ibm-modules/terraform-ibm-toolkit-vpn-server/runs/8123045343?check_suite_focus=true

Error: local-exec provisioner error
│
│   with module.vpn_module.null_resource.vpn_server,
│   on module/main.tf line 146, in resource "null_resource" "vpn_server":
│  146:   provisioner "local-exec" {
│
│ Error running command 'module/scripts/create-vpn.sh': exit status 1.
│ Output: API endpoint: https://cloud.ibm.com/
│ Authenticating...
│ OK
│
│ Targeted account GoldenEye GoldenEye Development's Account
│ (abac0df06b644a9cabc6e44f55b3880e) <-> 2314088
│
│ Targeted resource group vpc-count-rg
│
│ Targeted region eu-gb
│
│
│ API endpoint:      https://cloud.ibm.com/
│ Region:            eu-gb
│ User<EMAIL_ADDRESS>│ Account:           GoldenEye GoldenEye Development's Account
│ (abac0df06b644a9cabc6e44f55b3880e) <-> 2314088
│ Resource group:    vpc-count-rg
│ CF API endpoint:
│ Org:
│ Space:
│ ibmcloud is vpn-server-create    --name "vpc-count-rg-vpn-client2site"
│ --subnet
│ "0787-22955292-7290-43a2-b989-929f6dcb3b15,0797-51f502e2-05a8-473f-aa34-ff601a8a179c"
│ --cert
│ "crn:v1:bluemix:public:cloudcerts:eu-gb:a/abac0df06b644a9cabc6e44f55b3880e:ee96c507-6e85-4ec5-aaed-29478dfabb91:certificate:61017aae59eb2e2a3a0615c9218036ec"
│ --client-ca
│ "crn:v1:bluemix:public:cloudcerts:eu-gb:a/abac0df06b644a9cabc6e44f55b3880e:ee96c507-6e85-4ec5-aaed-29478dfabb91:certificate:fd7191b724c8e13fd9fd84b2bc850b54"
│ --client-ip-pool "<IP_ADDRESS>/16"    --client-dns "<IP_ADDRESS>,<IP_ADDRESS>"
│ --client-auth-methods "certificate"    --sg
│ "r018-1c72d75e-dcb3-44c2-9e0d-569c3b518a34"    --protocol "udp"    --port
│ "443"    --enable-split-tunnel "true"    --client-idle-timeout "600"
│ Migrate your vpn server certificates from Certificate Manager to Secrets
│ Manager and use the crn of the secret from Secrets Manager for the cert and
│ client-ca options, since the creation of vpn server with certificates from
│ Certificate Manager is deprecated.
│ Creating VPN server vpc-count-rg-vpn-client2site in resource group
│ vpc-count-rg under account GoldenEye GoldenEye Development's Account as
│ user<EMAIL_ADDRESS>│ FAILED
│ Failed to create a VPN server.
│
│ FAILED
│ Response HTTP Status Code: 404
│ Error code: vpn_server_certificate_not_found
│ Error message: The certificate
│ 'crn:v1:bluemix:public:cloudcerts:eu-gb:a/abac0df06b644a9cabc6e44f55b3880e:ee96c507-6e85-4ec5-aaed-29478dfabb91:certificate:61017aae59eb2e2a3a0615c9218036ec'
│ could not be found. Please try again with a correct certificate CRN.
│ More information:
│ https://cloud.ibm.com/docs/infrastructure/vpc/errors.html#vpn_server_certificate_not_found
│ Trace ID: 469588f0-4ae7-93b0-8f4f-ec5570bcf974
│
│
│ Failed to show VPN server.
│ Response HTTP Status Code: 404
│ Error code: not_found
│ Error message: Please check whether the resource you are requesting exists.
│ Error target name: ID, type: parameter
│ Error target value: vpc-count-rg-vpn-client2site
│ More information:
│ https://cloud.ibm.com/docs/vpc?topic=vpc-rias-error-messages#not_found
│ Trace ID: 12e4e575-c377-9e9e-8728-e9ce73f74e33
│
│ Creating VPN route vpc-network of server vpc-count-rg-vpn-client2site under
│ account GoldenEye GoldenEye Development's Account as user
│<EMAIL_ADDRESS>│ FAILED
│ Failed to create VPN route.
│
│ FAILED
│ Response HTTP Status Code: 404
│ Error code: not_found
│ Error message: Please check whether the resource you are requesting exists.
│ Error target name: ID, type: parameter
│ Error target value: vpc-count-rg-vpn-client2site
│ More information:
│ https://cloud.ibm.com/docs/vpc?topic=vpc-rias-error-messages#not_found
│ Trace ID: 78033f23-6a63-98ba-8984-446c4d5bfbcf
│
│
│ Creating VPN route services of server vpc-count-rg-vpn-client2site under
│ account GoldenEye GoldenEye Development's Account as user
│<EMAIL_ADDRESS>│ FAILED
│ Failed to create VPN route.
│
│ FAILED
│ Response HTTP Status Code: 404
│ Error code: not_found
│ Error message: Please check whether the resource you are requesting exists.
│ Error target name: ID, type: parameter
│ Error target value: vpc-count-rg-vpn-client2site
│ More information:
│ https://cloud.ibm.com/docs/vpc?topic=vpc-rias-error-messages#not_found
│ Trace ID: 40965354-0b51-9dad-adf4-081dc28766e3
│
│
│ Creating VPN route dns of server vpc-count-rg-vpn-client2site under account
│ GoldenEye GoldenEye Development's Account as user
│<EMAIL_ADDRESS>│ FAILED
│ Failed to create VPN route.
│
│ FAILED
│ Response HTTP Status Code: 404
│ Error code: not_found
│ Error message: Please check whether the resource you are requesting exists.
│ Error target name: ID, type: parameter
│ Error target value: vpc-count-rg-vpn-client2site
│ More information:
│ https://cloud.ibm.com/docs/vpc?topic=vpc-rias-error-messages#not_found
│ Trace ID: a595c556-4a62-9a0a-8215-e3e2c82f9e65
│
│
│
╵
Error: Process completed with exit code 1.

Need to create the Serviceauth since target cloud account does not have permission.

Added Service Auth but running into errors
│ If you have feedback on the design of this feature, please open a GitHub
│ issue to discuss it.
╵
╷
│ Error: External Program Execution Failed
│
│   with module.resource_group.module.clis.data.external.setup-binaries,
│   on .terraform/modules/resource_group.clis/main.tf line 22, in data "external" "setup-binaries":
│   22:   program = ["bash", "${path.module}/scripts/setup-binaries.sh"]
│
│ The data source received an unexpected error while attempting to execute
│ the program.
│
│ Program: /bin/bash
│ Error Message: igc release not found
│
│ State: exit status 1
╵
╷
│ Error: External Program Execution Failed
│
│   with module.vpn_module.module.clis.data.external.setup-binaries,
│   on .terraform/modules/vpn_module.clis/main.tf line 22, in data "external" "setup-binaries":
│   22:   program = ["bash", "${path.module}/scripts/setup-binaries.sh"]
│
│ The data source received an unexpected error while attempting to execute
│ the program.
│
│ Program: /bin/bash
│ Error Message: ibmcloud release not found
│
│ State: exit status 1

Resolved the issues as added latest container image


Propagate CostAllocationTag to EBS

Is your feature request related to a problem? Please describe.
Cost Allocation Tag is available but the tag is not being propagated to EBS volume of the launch template. It is important to have cost allocation tag being propagated into the volume as well to determine a more accurate cost attribution of infra.
Describe the solution you'd like
Under TagsSpecification of the stack launch template, we can add a ResourceType: volume as per the documentation and propagate UseCostAllocationTags, CostAllocationTagName, and CostAllocationTagValue into the EBS Volume of the launch template
Describe alternatives you've considered
Manually adding one into the EBS volume or creating the tag
Additional context
n/a

hey, this is a great idea - would you be willing to submit a PR for it?


website icons is missing

hello
greetings for you all
first of all big thanks for the team of ph7cms script
i'm using nginx and was able to set it up, everything seems to work fine..
there is little issue i have which is
photos icon not showing, photo icon for members who doesn't have photos, i mean the default icons in homepage and inside website too
uploaded photo explain what i mean

also admin setting for  homepage SEO text is not showing in the admin page, can't edit or change it, blank white in the SEO text section

Why are you using Nginx? As it says in the documentation you may have to add more rules to your nginx configuration to get this to work properly. I have very little experience with nginx so unable to elaborate!

Why are you using Nginx? As it says in the documentation you may have to add more rules to your nginx configuration to get this to work properly. I have very little experience with nginx so unable to elaborate!
thank you for your quick reply @Undefined-Variables
i'm using nginx due to its light-weight resource utilization

@Undefined-Variables good evening, i tried with apache server i get 500 internal error just after uploading files to root, i checked your docs and make sure i have my files permissions set right
and PHP config set right, where i can check error log? i can't find anything in apache log, what do you suggest?
thank you

the fix for my issue was deleting everything in htaccess file and put the following code
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

@zeze6 Yeah I was just thinking as I read your post an initial 500 is normally to do with .htaccess file! Most of the rules in the example supplied for your convenience should be in your apache configuration (httpd.conf or apache2.conf depending on version you are running) it’s more efficient to have them in apaches configuration as the rules are read and set upon apache start.
Having them in the .htaccess means they are called on every single request! This quickly consumes resources when 100 users are all browsing the site!
Unless you have experience using nginx and fully understand the way the rules are applied I’d recommend sticking with Apache as the software has been developed and tested with it! When problems arise you have more chance of getting help.
I’m glad you worked it out and thank you for coming back and sharing your solution, this helping and getting help is a two way street :)
By the way @pH-7 is the led developer on this project, I simply help out with issues as a way of saying thanks for all his work...give and take!

@Undefined-Variables thank you so much for explaining this to me
i have another issue, sometimes when users clicks on one of the pages for example, browse or photo a new tab opens and redirect to domain name IP like this www.xxx.xx.xxx.xxx/photo-gallery x is the IP for my domain, not domainname.com
it doesn't happen always but it happens a lot, any suggestions?

You can use a rewrite rule to stop this. Something like...
RewriteCond %{HTTP_HOST} ^xxx.xxx.xxx.xxx$ RewriteRule ^(.*)$ https://your-real-domain.com/$1 [R=301,L]
This should solve your issue, as for the course... Its down to your servers configuration.

@Undefined-Variables thank you the issue still not solved

This rule is perfect for the problem you have... Is mod_rewrite installed?

Open terminal type this...
sudo a2enmod rewrite

then
sudo service apache2 restart

@Undefined-Variables thank you i forgot to restart apache it works now, but website still open new tab whenever this happens

I’m pleased it works but I am going to leave you to solve this one :)

@Undefined-Variables haha brother i'm not coder by any means, i just know the basic things and since i have unmanaged VPS with no support included i learned from internet and from great people like you how to fix everything by my self, don't leave me please

@zeze6 Try removing the L from the rule I gave you.
RewriteRule ^(.*)$ https://your-real-domain.com/$1 [R=301,L]
To
RewriteRule ^(.*)$ https://your-real-domain.com/$1 [R=301]
There are more redirects ahead in the file so the L is not required yet!

the fix for my issue was deleting everything in htaccess file and put the following code
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

Plus you have removed half the rules you need in the htaccess file. I would upload the complete one and add the rules above and if you get a 500 remove one rule at a time, if its still giving 500 replace the rule and try the next one!

@Undefined-Variables thank you for your suggestion, i put back the htaccess file and i removed the following code and kept the one i put earlier, no errors
<IfModule mod_rewrite.c>
    <IfModule mod_negotiation.c>
        Options -MultiViews -Indexes
    </IfModule>
    Options +FollowSymLinks

    RewriteEngine On

    ## Uncomment the below "#RewriteBase /" (remove "#") if pH7CMS is installed in a folder, and add the folder name after the slash "/"
    ## Example: If pH7CMS is installed in "/public_html/ph7cms/", then it should be "RewriteBase /ph7cms/", however it may depends of your Apache configuration.
    ## If you need, please ask your hosting company or check the Apache doc: http://httpd.apache.org/docs/current/mod/mod_rewrite.html

    # RewriteBase /
</IfModule>

Wonderful :) If you issue is solve you can close this issue :)

@Undefined-Variables ok i will but i have question please, suddenly i can't login in admin panel and can't reset password not receiving email is there a way to change or view the password?
thank you so much brother

@Undefined-Variables i tried with chrome and i was able to login but couldn't do it on firefox

Absolutely not!!! Passwords are hashed then stored in the database! Any other way is just asking for problems! Sounds like Firefox is not allowing cookies, I don't have the time to look right now! But that is where I would start looking...

oh i see, but now it won't let me be able to solve the Captcha for admin and users too

also i haven't set cron job yet :) i tried to do so i get 403 error

the Captcha worked when i used windows 10 but did not work on Ubuntu :) ok my man @Undefined-Variables when you come back please tell me what are the correct settings for cron, have a good night

This is the 4th issue now on this ticket, we are not here to provide step by step support for every issue you see and you are asking for help with things that are explained in the documentation
https://ph7cms.com/doc/

Your cron tab should look something like this
0 0 * * * curl "https://your.domain.here.com/asset/cron/96h/Database/?secret_word=your_secret_string_here"
0 0 * * * curl "https://your.domain.here.com/asset/cron/96h/General/?secret_word=your_secret_string_here"
0 0 * * * curl "https://your.domain.here.com/asset/cron/96h/Banned/?secret_word=your_secret_string_here"

I am busy im sorry I can't help further :)

when i try to put GET http://domain.com/assets/cron/96h/Database/?secret_word=mysecretword
i get 403, i looked at htaccess and found out there was some lines cause this error lines are
</Limit>

ErrorDocument 400 /error/http/index?code=400
ErrorDocument 401 /error/http/index?code=401
ErrorDocument 402 /error/http/index?code=402
ErrorDocument 403 /error/http/index?code=403
ErrorDocument 404 /error
ErrorDocument 405 /error/http/index?code=405
ErrorDocument 500 /error/http/index?code=500
ErrorDocument 501 /error/http/index?code=501
ErrorDocument 502 /error/http/index?code=502
ErrorDocument 504 /error/http/index?code=504
ErrorDocument 505 /error/http/index?code=505

# URL Rewrite

when i remove these lines and put the same cron job, it works! i don't know if this is misconfiguration in the script or bug
are these all the cron list? in this link https://ph7cms.com/doc/en/cron-list

sorry @Undefined-Variables i just saw your comment, and yes i read the doc but i thought maybe the docs were old i just want to make sure, thank you so much for being so helpful i really appreciate that

Not a problem, close this ticket and open a new one for any future problems, this helps others find issues in the future. Best of luck!

@Undefined-Variables ok Sir :)


New user has admin status right after registration

Today I noticed that recently registered user has admin privilege that was quite odd.
I've created new temp account to test behaviour and, yep, it has default status as expected. So it's totally unclear to me what could it be.

We had a bug with that recently, so you're not imagining problems here.  I think @timabbott can help you with a permanent fix.

I assume this was the bug fixed with Zulip 2.1.5. Quoting the changelog:

CVE-2020-14215: Fixed buggy 0198_preregistrationuser_invited_as
database migration from the 2.0.0-rc1 release, which incorrectly added
the administrator role to invitations.
CVE-2020-14215: Added migration to clear the administrator role from
any invitation objects already corrupted by the buggy version of the
0198_preregistrationuser_invited_as migration.

@FoggyFinder can you confirm?

Let's count as fixed. I'll reopen issue if I see it again.


Incorrect type for vk_sys::Result

The type of vk_sys::Result is u32. But error values are negative. Shouldn’t it be i32?

vk_sys has been deprecated for a while, and was removed from the Vulkano repository. I think you may be looking at outdated code somewhere?

Well, crates.io still says that this is its repository, so this is where I reported the issue.

I see, well it's being misleading then, sorry about that. @AustinJ235 @Eliah-Lakhin could that be removed from crates.io or marked as deprecated in some way?

From what I know it is not easy to remove a crate from crates.io.
Here some solutions:
https://www.reddit.com/r/rust/comments/p1u5hp/removing_my_own_unused_crate_from_cratesio/
IIRC the reason is to be found historically in a mess that happened at npm:
https://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm

I don’t think it should be removed anyway; it’s not unused. Maybe add a README that informs people of the deprecation?

We marked https://github.com/vulkano-rs/vk-sys source repository as archived and deprecated.

@Eliah-Lakhin The crates.io entry doesn't point to that repository though. Could that be changed?

I don't think we can change already published versions. Maybe we can publish another one just with updated link.


Can't preprocess dataset using meta-llama/Meta-Llama-3.1-8B model

Please check that this issue hasn't been reported before.

[X] I searched previous Bug Reports didn't find any similar reports.

Expected Behavior
I expected to have a pre-processed dataset after running python -m axolotl.cli.preprocess
Current behaviour
I get this error:
NotImplementedError: aten::_local_scalar_dense: attempted to run this operator with Meta tensors, but there was no abstract impl or Meta kernel registered. You may have run into this message while using an operator with PT2 compilation APIs (torch.compile/torch.export); in order to use this operator with those APIs you'll need to add an abstract impl.Please see the following doc for next steps: https://docs.google.com/document/d/1_W62p8WJOQQUzPsJYa7s701JXt0qf2OfLub2sbkHOaU/edit

Full trace:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/wgilliam/development/projects/axolotl/src/axolotl/cli/preprocess.py", line 96, in <module>
    fire.Fire(do_cli)
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/fire/core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/fire/core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/development/projects/axolotl/src/axolotl/cli/preprocess.py", line 85, in do_cli
    AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3788, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 1068, in __init__
    self.model = LlamaModel(config)
                 ^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 845, in __init__
    [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)]
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 845, in <listcomp>
    [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)]
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 632, in __init__
    self.self_attn = LLAMA_ATTENTION_CLASSES[config._attn_implementation](config=config, layer_idx=layer_idx)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 306, in __init__
    self.rotary_emb = LlamaRotaryEmbedding(config=self.config)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 119, in __init__
    inv_freq, self.attention_scaling = self.rope_init_fn(self.config, device, **self.rope_kwargs)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/transformers/modeling_rope_utils.py", line 330, in _compute_llama3_parameters
    if wavelen < high_freq_wavelen:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wgilliam/mambaforge/envs/axolotl/lib/python3.11/site-packages/torch/utils/_device.py", line 78, in __torch_function__
    return func(*args, **kwargs)
    ```

### Steps to reproduce

Run > `python -m axolotl.cli.preprocess`

### Config yaml

```yaml
base_model: "meta-llama/Meta-Llama-3.1-8B"
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

seed: 9
data_seed: 9

hub_model_id:
wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

datasets:
dataset_prepared_path: data/last_run_prepared
val_set_size: 0.05
output_dir: outputs

sequence_len: 3072
sample_packing: false
eval_sample_packing: false
pad_to_sequence_len: false

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

adapter: qlora
load_in_8bit: false
load_in_4bit: true
strict: false

lora_model_dir:
lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:

# NOTE: If you add token you will need to save thse lora modules
# lora_modules_to_save:
#   - embed_tokens
#   - lm_head

gradient_accumulation_steps: 4 # 8
micro_batch_size: 2 #1
eval_batch_size: 2
num_epochs: 3
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 3e-5 #0.0002
# max_grad_norm: 1.0
# adam_beta2: 0.95
# adam_epsilon: 0.00001
# save_total_limit: 12

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
s2_attention:

loss_watchdog_threshold: 5.0
loss_watchdog_patience: 3

# max_steps: 100
# warmup_steps: 100 #10
warmup_ratio: 0.2
evals_per_epoch: 4 #8
# eval_steps: 10
eval_table_size:
eval_max_new_tokens: 512 #128
save_total_limit: 1
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
  pad_token: <|end_of_text|>

# NOTE: If you add tokens, in addition to updating the lora config (see above), you'll
#       likely need to reduce your overall batch size if training on a GPU poor rig
# tokens:
#   - <function-definitions>
#   - </function-definitions>
#   - <function-thoughts>
#   - </function-thoughts>
#   - <function-calls>
#   - </function-calls>
#   - function_call

save_safetensors: true

### Possible solution

N/A

### Which Operating Systems are you using?

- [X] Linux
- [ ] macOS
- [ ] Windows

### Python Version

3.11

### axolotl branch-commit

main

### Acknowledgements

- [X] My issue title is concise, descriptive, and in title casing.
- [X] I have searched the existing issues to make sure this bug has not been reported yet.
- [X] I am using the latest version of axolotl.
- [X] I have provided enough information for the maintainers to reproduce and diagnose the issue.

I don't see a dataset in your configuration YAML. Did you redact it? Can you provide some info on the dataset/prompt type you're trying to preprocess?

Sorry, I'm adding that in dynamically (running in jupyter). I'm using the template free format which is working fine in old Llama3 ...
# axo_config_fpath = "configs/axolotl_configs/llama3-8b-qlora.yaml"
axo_config_fpath = "configs/axolotl_configs/llama3.1-8b-qlora.yaml"

train_data_fpath = "data/train_reviewed_template_free_1000.jsonl"
train_data_config = str(f'[{{"path": "{train_data_fpath}", "type":"input_output"}}]')

python -m axolotl.cli.preprocess {axo_config_fpath} --datasets '{train_data_config}'

Closing this out.  Can verify that pip install from the transformers main branch provides the necessary fix.


Add NL and EU logo

@arildwtv

I'll fix this.

I propose an easier solution. What if we only use the virus icon svg and render the text part (Coronastatus.no/.nl/etc) in HTML? That way we don't need SVGs for every country. @fossecode

Yeah thats a good idea. Makes it less tedious to add more countries. Could you create the logo without TLD @arildwtv?

On it.

You could use something like this:

But feel free to do whatever you want :)

@adriaanvanrossum Until we decide on a final design, I'll just "clone" the current logo we're using, hope that's ok. :)
@fossecode Where do we provide the tld parameter? Should we define it in example.json as TLD? We already have a LANGUAGE config parameter, but these aren't necessarily one-to-one.

https://github.com/BustByte/coronastatus/pull/217/files


Mind Flay and Trading

By the consequences of your mind flay activities being seen, does it mean you don't get to trade at all anymore? I'm not 100% positive it is a problem with your fine mod, but its one of the things I interacted with last before I saw the change. Interacting with villagers has them shake their head at me, as if they don't have a profession and doesn't open the trade GUI.
I can however, trade with Wandering Travelers; but with villagers? Nope. No matter what village, no matter if I got hero of the village, can't be done. Is this something your mod has the potential to bug out on at all?

That's really strange. I don't know why that would happen - we just call the same method that's called when you murder a villager

Any update on this?

Apologies for not responding sooner; I must have missed the initial reply.
I believe it was ultimately not the cause of hexcasting. Sorry for the trouble, though I appreciate the check-in to be sure.


Neither local flows work

Referencing docs/oauth-installed.md and https://developers.google.com/sheets/api/quickstart/python
Both flows result in errors.
flow.run_local_server results in a AttributeError: 'NoneType' object has no attribute 'replace' error
flow.run_console results in AttributeError: 'InstalledAppFlow' object has no attribute 'run_console'
Environment details

OS type and version: AlmaLinux 8
Python version: 3.8.2
pip version: 23.1.2
google-api-python-client version: 2.88.0

Steps to reproduce

Try to connect use a local OAuth flow (see either references at the top)
?

Code example
flow = InstalledAppFlow.from_client_secrets_file(creds_path, SCOPES)
creds = flow.run_local_server(port=8080)

flow = InstalledAppFlow.from_client_secrets_file(creds_path, SCOPES)
creds = flow.run_console()

I see flow.run_console has been removed completely.
The problem I was getting with run_local_server is being triggered by the fact that handle_request respects socket.getdefaulttimeout() over the timeout value being passed.

I see flow.run_console has been removed completely.
The problem I was getting with run_local_server is being triggered by the fact that handle_request respects socket.getdefaulttimeout() over the timeout value being passed.

The issue with that is that run_local_server  requires a webserver to be installed.   How will this work on colab?


inject is not working as a reactive function in version 4.1.8

According to this resolved issue, primitives should be tracked inside @inject blocks, and be 'reactive', as of mobx-react version 4.0.0
I've created an example where this doesn't work correctly using version 4.1.8.
It doesn't work where a primitive is destructured inside the @inject block before being passed in as props.
In the following examples, selecting a value in the dropdown calls:
setFormFilter(event.target.value); //update the form id in the store.
renderResponse(); //calls a function in the parent which should have access to the updated formFilter value from the store.
Passing the whole state object from store works.
https://codesandbox.io/s/1r7YM60lR
componentWillReact is called immediately after setFormFilter(), and before renderResponse().
Destructure the primitive formFilter inside the stores state doesn't work.
https://codesandbox.io/s/9QjVnKq1Z
componentWillReact is called after both the setFormFilter(), and renderResponse() calls.

Can you clarifiy your example a bit? Some remarks:
  const { setFormFilter, state: responseState } = store;. store.state doesn't seem to contain a responseState field? So I am not sure what is supposed to happen there
extendObservable(this.state, {
      formFilter: formFilter,
    });

doesn't do anything special, you cansimply write this.state.formFilter = formFilter
I would appreciate it if the example is simplified; it is currently pretto to hard to follow, and it is a bit unclear to me what you are trying to acieve in the first place :wink:
For example what is a callback like renderResponse doing? Your not supposed to trigger rendering yourself in the first place

(renderResponse() is just an method in the parent Component. It doesn't force re-render.)
The issue is that when a state property is deconstructed in the inject block,
For the 2 examples above, open the browser console, change the dropdown value and compare the output. Depending on whether you pass the whole state vs Destructuring the state, the store values do not update at the same point in time.
The first example (pass whole state): store.formFilter is 1 //correct
The second example (pass destructured state): store.formFilter: -1 //incorrect.

As said, your store is strangely set up, as state is initially not an observable property, but you make it an observable property only when the setFormFilter function is called for the first time. Changed that and now both examples provide the same output (although the first has the same erronous setup as the second one had). https://codesandbox.io/s/jE7RrZy
If you have follow up questions, feel free to reopen, please mention what you expect the output to be, and why.

state is initially not an observable property, but you make it an observable property only when the setFormFilter function is called for the first time.

Does my original code (below) not initialise state as an observable?
state = observable({
    formFilter: -1,
});

@robinspark your first example makes { formFilter } object observable, but not state. So changing the .formFilter attribute would be picked up, but a reassignment to the state property doesn't. Unlike the second approach which makes both observable. (@observable is like extendObservable(this, { state: { formFilter }})


💡 [REQUEST] - Replace Black, isort, Pylint and Flake8 with Ruff

Summary
Ruff is "an extremely fast Python linter and code formatter, written in Rust", and a drop-in replacement for Flake8, isort and Black. It also has an overlap with Pylint.
Basic Example
Ruff also integrates with Visual Studio code through an official extension.
Screenshots
No response
Drawbacks
None I can think of.
Unresolved questions
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

TODO: Update CONTRIBUTING.md, and mention the Ruff extension in VS Code. Ruff is so fast that it can be used as an LSP, in watch mode.


Security Advisories: CWE and CVSS

Summary
Today, GitHub Security Advisories enable maintainers to privately discuss, fix, and disclose information about vulnerabilities in their projects. This information is used to automatically trigger security updates for participating GitHub repositories. The updated security advisory form will make it easier to provide details required for a CVE.
Intended Outcome

Maintainers can communicate severity using the CVSS calculator or by providing a CVSS formula from their own calculator
Maintainers can specify the Common Weakness Enumeration (CWE) that categorizes their vulnerability

How will it work?
During the creation of a Security Advisory, the maintainer will be able to add the CWE and CVSS score.

https://github.blog/changelog/2021-02-09-cwe-and-cvss-in-security-advisories/


Fit to view

https://user-images.githubusercontent.com/1093738/130471287-2c9a632a-5af9-43ff-8b3b-061da34be7d7.mp4
Todo:

[ ] Find appropriate icon

@davidkpiano Could be working better on the vertical axis? A bit more space needed at the bottom

For an icon, any of these font-awesome ones:
Compress
Expand
Expand-Alt
Expand-Wide
Expand-Arrows
Compress-Arrows-Alt


Add an option to bytecode compile during installation

Add a --compile option to pip install and pip sync.
I chose to implement this as a separate pass over the entire venv. If we wanted to compile during installation, we'd have to make sure that writing is exclusive, to avoid concurrent processes writing broken .pyc files. Additionally, this ensures that the entire site-packages are bytecode compiled, even if there are packages that aren't from this uv invocation. The disadvantage is that we do not update RECORD and rely on this comment from PEP 491:

Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD.

If this is a problem we can change it to run during installation and write RECORD entries.
Internally, this is implemented as an async work-stealing subprocess worker pool. The producer is a directory traversal over site-packages, sending each .py file to a bounded async FIFO queue/channel. Each worker has a long-running python process. It pops the queue to get a single path (or exists if the channel is closed), then sends it to stdin, waits until it's informed that the compilation is done through a line on stdout, and repeat. This is fast, e.g. installing jupyter plotly on Python 3.12 it processes 15876 files in 319ms with 32 threads (vs. 3.8s with a single core). The python processes internally calls compileall.compile_file, the same as pip.
Like pip, we ignore and silence all compilation errors (https://github.com/astral-sh/uv/issues/1559). There is a 10s timeout to handle the case when the workers got stuck. For the reviewers, please check if i missed any spots where we could deadlock, this is the hardest part of this PR.
I've added uv-dev compile <dir> and uv-dev clear-compile <dir> commands, mainly for my own benchmarking. I don't want to expose them in uv, they almost certainly not the correct workflow and we don't want to support them.
Fixes #1788
Closes #1559
Closes #1928

The disadvantage is that we do not update RECORD and rely on this comment from PEP 491:

Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD.

If this is a problem we can change it to run during installation and write RECORD entries.

I would validate that Pip successfully uninstalls the pyc files, because unless I am misunderstanding this issue it probably doesn't: https://github.com/pypa/pip/issues/11835

I ran
cargo run venv --seed; cargo run pip install --compile tqdm
.\.venv\Scripts\pip uninstall -y tqdm

and the tqdm folder was fully removed.

On second thought, I’m wondering if this should just be a stand-alone command. Since it runs over the whole env, there’s no reason it needs to be coupled to install / sync.

We can expose it as uv compileall (we can copy the dev command over), my only counterargument is not extending our external interface more. I'd like to keep the --compile flag either way to keep the install operation a single command.
Fwiw there's also python -m compileall, it does the same thing just slower:
$ hyperfine --prepare "uv venv && uv pip install jupyter plotly" "python -m compileall .venv/lib/python3.12/sit
e-packages/"
Benchmark 1: python -m compileall .venv/lib/python3.12/site-packages/
  Time (mean ± σ):      5.407 s ±  1.026 s    [User: 4.354 s, System: 0.427 s]
  Range (min … max):    4.668 s …  7.699 s    10 runs

I do think the better approach here is to deal with problems with byte-code compiling at the level of the subprocess. And specifically, only use timeouts when something you know could legitimately hang (like, say, killing a subprocess).

The problem i'm mainly thinking about is when the python process is in some regard broken, e.g. image python is replaced by a bash script that does > /dev/null (this isn't the case, but i've seen pythons that were not pythons or behaved strangely for some other reason).  I'm less worried about OS operation such as killing a subprocess.
The second problem i'm thinking about is e.g. a panic in one of the threads that makes tokio behave oddly or a concurrency bug on our end, even i don't have good insight about the failure modes here.
I feel this is a trade-off we have to make: We can put no timeout and expect the user to kill the process (or the CI to timeout) vs. we specify and unreasonably high limit after which we abort (similar to the timeout we put on http requests). If we want to put this in control of the user, i would remove the timeouts.

The problem i'm mainly thinking about is when the python process is in some regard broken, e.g. image python is replaced by a bash script that does > /dev/null (this isn't the case, but i've seen pythons that were not pythons or behaved strangely for some other reason). I'm less worried about OS operation such as killing a subprocess.

I think for cases like this, I'd be happier with a large timeout at a more granular level. e.g., a ~60 second timeout within the worker. So after 60 seconds of no output, the worker would kill the subprocess and report an error to the user. The 60 second wait isn't great, but the user will at least eventually get some kind of error message instead of a truly indefinite hang.
If this were a common failure mode then this wouldn't be great. But if a Python process is as badly broken as > /dev/null, I think I'm more-or-less okay with having to wait longer before saying something to the user.
And since the timeout is applied more granularly (at the level of a single subprocess), you could feasibly decrease it.

The second problem i'm thinking about is e.g. a panic in one of the threads that makes tokio behave oddly or a concurrency bug on our end, even i don't have good insight about the failure modes here.

I don't have as much experience with panics in Tokio, but I think it will catch them and the task will quit and you'll get a JoinError. But I defer to you on this one. You probably have more practical hands-on experience than I do here.

I feel this is a trade-off we have to make: We can put no timeout and expect the user to kill the process (or the CI to timeout) vs. we specify and unreasonably high limit after which we abort (similar to the timeout we put on http requests). If we want to put this in control of the user, i would remove the timeouts.

Yeah to be clear, I think it's fine to have timeouts somewhere. I'm mostly more uneasy with having them at a higher level. But it could all be fine. I just don't know.

This LGTM, though I'm also curious if we can move the timeout down a level?

I moved the temporary directory into the cache just to be safe. I'll merge this to keep it moving.

I moved the temporary directory into the cache just to be safe.

What's the problem we're avoiding this way?


I want Aurora to only work when a specific process is running.

I want Aurora to only work when Overwatch is running.
I use Aurora to take advantage of Overwatch's chroma lighting.
I check the profile of Corsair keyboard through lighting.
so it's hard to use Aurora all the time.
So I only run Aurora when I'm playing Overwatch, which is very inconvenient.
It would be nice to have the ability to set Aurora to only work when certain processes are running.

Sorry for really late response but I've implemented this on latest release
https://github.com/Aurora-RGB/Aurora/releases/tag/v145


Distro features

Readiness

[x] Merge (pending reviews)

Overview
Include how to change (append and remove) and examples for DISTRO_FEATURES, IMAGE_FEATURES, and MACHINE_FEATURES.
any idea on what else to add?

Docs for b44a06326fc69a4fb476b025ade72a9d7c30a34a are browsable at: https://ci.foundries.io/projects/fio-docs/builds/2275/docs/artifacts/html/index.html

At the moment, I don't know what else that may be appropriate to add, but I will let you know if anything comes to mind later.

Docs for 35fca6bb06f9a2a2da98c6b58cfbd0b70b685211 are browsable at: https://ci.foundries.io/projects/fio-docs/builds/2276/docs/artifacts/html/index.html

Docs for dfa4ecbd6e33a90350b9e7a9a8c8d29d75845e36 are browsable at: https://ci.foundries.io/projects/fio-docs/builds/2277/docs/artifacts/html/index.html

@angolini shall I go ahead and merge?
@kprosise yes, please!


Docs for new config file params

Description of the Change
Added descriptions of the following params:

mst_expiration_time
max_rounds_delay
stale_stream_max_rounds

Benefits
doc coverage

Possible Drawbacks

@neewy
I am not sure that both suggested changes are applicable at the moment. There were no load tests for MST transactions yet, and testing for 500ms delay is still in progress.


Publish 09/07/2021, 10:30 AM

Docs Build status updates of commit dc300d4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2019-ps/printmanagement/Set-PrintConfiguration.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2019-ps/startlayout/Export-StartLayout.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


doesn't retry Google Cloud Storage "end of file before message length reached"

Error: could not POST https://storage.googleapis.com/upload/storage/v1/b/mybucket/o?uploadType=media&ifGenerationMatch=0&name=myfile.csv
caused by: error sending request for url (https://storage.googleapis.com/upload/storage/v1/b/mybucket/o?uploadType=media&ifGenerationMatch=0&name=myfile.csv): error from user's HttpBody stream:
request or response body error:
request or response body error:
error reading a body from connection:
end of file before message length reached
0: failure::backtrace::internal::InternalBacktrace::new
1: <failure::backtrace::Backtrace as core::default::Default>::default
2: <core::result::Result<T,E> as failure::result_ext::ResultExt<T,E>>::with_context
3: dbcrossbarlib::clouds::gcloud::storage::upload_file::upload_file::{{closure}}
4: <std::future::GenFuture<T> as core::future::future::Future>::poll
5: futures_util::stream::stream::StreamExt::poll_next_unpin
6: <S as futures_core::stream::TryStream>::try_poll_next
7: <futures_util::stream::try_stream::try_collect::TryCollect<St,C> as core::future::future::Future>::poll
8: <std::future::GenFuture<T> as core::future::future::Future>::poll
9: dbcrossbar::cmd::cp::run::{{closure}}
10: <std::future::GenFuture<T> as core::future::future::Future>::poll
11: <futures_util::future::maybe_done::MaybeDone<Fut> as core::future::future::Future>::poll
12: <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
13: <std::future::GenFuture<T> as core::future::future::Future>::poll
14: std::thread::local::LocalKey<T>::with
15: tokio::runtime::basic_scheduler::BasicScheduler<P>::block_on
16: tokio::runtime::context::enter
17: tokio::runtime::handle::Handle::enter
18: dbcrossbarlib::tokio_glue::run_futures_with_runtime
19: dbcrossbar::main
20: std::rt::lang_start::{{closure}}
21: std::rt::lang_start_internal::{{closure}}
          at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/rt.rs:52
    std::panicking::try::do_call
          at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/panicking.rs:305
22: __rust_maybe_catch_panic
          at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libpanic_unwind/lib.rs:86
23: std::panicking::try
          at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/panicking.rs:281
    std::panic::catch_unwind
          at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/panic.rs:394
    std::rt::lang_start_internal
          at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/rt.rs:51
24: main

This looks like some kind of dropped connection or something. All our HTTP machinery is handled by pretty solid low-level libraries, IIRC. But we open up so many connections against Google APIs that even rare weirdness on Google's end can break things.

This looks like some kind of dropped connection or something. All our HTTP machinery is handled by pretty solid low-level libraries, IIRC. But we open up so many connections against Google APIs that even rare weirdness on Google's end can break things.


Sidebar / a11y zoom / smallscreen / mobile / tablet

Makes Headlamp more responsive

zoom + 400% seems to work
More usable with mobile, and tablet sized browsers
App has been broken up in smaller components

Apart from the side bar also the tab navigation, and SimpleTable is responsive.
How to use
Try using at +400% zoom
Try using in a mobile, and tablet views
Testing done
Tested loading with different sizes and using the website.
Tested zooming to different sizes and using it.
Screenshots

Zoom 400%
xs (mobile)
medium (tablet)

Zoom 400%
You can see here that it uses the xs (mobile) views when zoomed in so much.

xs (mobile size) view
Note the table can be scrolled around, and there is a toolbar menu as well.
The headlamp icon on the left opens the side bar drawer (see below).

xs (mobile size) view temporary drawer open for side bar

medium (Tablet sized view)
Note the tab navigation is all fitting within the screen and there are arrows to use the extra tabs.

Future work
The loading screens on mobile view could be improved, but they are usable... so probably will address in another PR another time.
Note: Mostly tested with chrome/ff responsive testing tools... not with real mobile devices. So there are probably minor issues on some devices.
Only one extra storybook set of stories was made which was useful for testing a component. The other extracted components from the big App.tsx didn't get extra stories added in this PR.

Hi @illume . Nice work, I like how you reduced the sidebar to a single icon/button on small screens.
Here are two issues I think should be fixed before merging:

Some dialogs close immediately after opening; this seems to happen when the dialog is initiated from the sidebar on "phone mode", i.e. seems to be related with the sidebar getting closed. I tried to record a gif but my gif app is already in Feierabend. But choosing any aspect ratio where the sidebar is hidden by default and clicking in the + icon or the version button will reproduce this for me.
I think some dialogs should be full screen when on small screens. I'd say that any dialog that occupies >50% of the screen on small devices should be made fullscreen in that resolution. Fullscreen should help with cases like this a bit:

As you noted, the cluster chooser dialog looks very broken but we have to redesign it anyway because it looks bad if there are more than 3 clusters available. (And this should be a different PR)

@joaquimrocha thanks for having a look! I'll have a look at these issues.

The links to dialogs at the bottom of the sidebar do not close the drawer anymore.
All dialogs have been fixed on small view, and the editor one made full screen.
labels were made to text wrap so they don't grow outside of the grid (in small views especially)

Screenies below...

Put in fixes for those comments, and rebased on main.


Reduce the amount of labels for Flutter Repos

Reduce the amount of labels used in Flutter Repos
Problem
The flutter/flutter repo alone has 393 labels, a lot that are no longer in use.
Collaboration
A document has been created to begin collaborating on a solution to this problem

cc @jmagman


Update VMMC vs HIV testing code ordering

It would be more logical, with respect to the timeline of events in reality, to update VMMC after HIV testing so that people about to undergo VMMC are tested first. This way we can rely on testing and diagnosis for the current time step when selecting the VMMC sub-population. This would also mean we need to somehow mark people for VMMC before they go through the procedure and that VMMC does not affect sexual behaviour until the next time step.

Yes I agree, although I don't think VMMC affects sexual behaviour, but if course it HIV acquisition risk.


Add IsCommutativeMonoid' typeclass

Description
This sits between CommutativeMonoid and IsCommutativeMonoid. It doesn't bundle the carrier but everything else. It is a lot better behaved as a typeclass than the other two, and enables a bunch of cleanup.
Checklist

[x] Commit sequence broadly makes sense and commits have useful messages
[x] Code is formatted according to CONTRIBUTING.md
[x] Self-reviewed the diff

@williamdemeo the issue here is that the monoid stuff that's defined in the stdlib is unsuitable for typeclass resolution. For typeclass resolution to behave nicely, we need to have the carrier as an argument to the class, but we don't want to pass the rest of the structure (since that's what the typeclass is supposed to provide). But the stdlib only has fully bundled (no arguments) and fully unbundled (everything except the laws as arguments) versions.


TFBuild.TFBuildCommands API Documentation

What You Are Seeing?
The TFBuildCommands API documentation is very light.
What is Expected?
Code samples
Details of expected results/outputs in TFS
Versions of TFS supported
What environment are you running on?  Windows? Linux? Mac?
Windows
Are you running on a CI Server?  If so, which one?
TFS 2017.1

I'll see what I can do about adding some examples, but a lot of the documentation is straight from what little info Microsoft is providing 😄

that would be awesome, shame about poor MS documentation.

Just came across this, https://github.com/agc93/Cake.BuildSystems.Module
Seems to do about everything i need, so maybe can save you some time updating the documentation
Maybe just reference this module somewhere else on the site, unless its already there and i missed it.

@pcoombe I wrote that module as well so familiar with its operation! 😁 I need to make a couple of other changes as well so I will update docs at the same time.


Inverse: pedestrian shortcuts

https://www.openstreetmap.org/way/671146174 as a motivating example.
Look for a driving route between the endpoints of a footpath, and calculate the detour ratio. High values are better pedestrian shortcuts. (Complications include having to take into account all the complexities of driver routing, like turn restrictions and which direction to do the routing. And even in this example, both endpoints of the footpath aren't driveable -- have to search further on the graph.
An analogous route / score mode would be helpful. And showing both scores in one place could be interesting, to find areas favoring one mode.

Closer to this, but it's weird in a few ways.

This anecdote isn't showing up as dramatically, because I think https://www.openstreetmap.org/way/83592298 is missing restrictions.
Anyway,

[ ] Think through "mode comparison" approaches. For an individual route, compare directness. In score mode, have a way to look for places where detour factor is very different for two modes.


Added set_len method to Buffer

Adding set_len enables to change the length value of the buffer.
This is required if get_mut changes the length of the buffer.
I think it will be great to add generic vector method such as extend_from_slice, copy_from_slice, with the according length change. I can add it to this pull request if interested.

Thank you for the PR! 🙇  Indeed this is needed in some cases.

Note that Buffer is primarily an immutable container (think Arc<Vec<>>), this is why we do not expose mutable methods to it (other than returning the mutable vector ^^)


Re-organize mainloop and interrupt operations

fix #44
#53?

seems working fine on the real system for a few weeks


Enable skipLibCheck: false to work with Magic SDK + TypeScript projects

:rocket: PR was released in<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>:rocket:

:rocket: PR was released in<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>:rocket:

:rocket: PR was released in<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>:rocket:


'MathF' does not exist in the current context

yup. another problem lol. how can it not find math???
logan@MacBook-Air celeste-ios % sudo '/Users/logan/Desktop/celesteios/celeste-ios/build.sh'
Password:
~ Celeste iOS build script ~
written by rooot
https://github.com/RoootTheFox/celeste-ios
This project has been tested on the Linux Steam build of Celeste <IP_ADDRESS> but SHOULD work on any other FNA-based Celeste build
Enter the full path to an UNMODIFIED, FNA-based (!) Celeste install: /Users/logan/Library/Application Support/Steam/steamapps/common/Celeste/Celeste.app/Contents/Resources
(info) verifying celeste install
(info) found steam build of celeste
(info) fmod found
(info) checking for required tools...
(info) cloning submodules...
(info) https://github.com/FNA-XNA/FAudio already cloned in lib/FAudio
(info) https://github.com/FNA-XNA/FNA3D already cloned in lib/FNA3D
(info) https://github.com/flibitijibibo/SDL2-CS already cloned in lib/SDL2-CS
(info) https://github.com/FNA-XNA/Theorafile already cloned in lib/Theorafile
HEAD is now at 2b8d237 Support SDL2-CS.Settings.props for both project types
HEAD is now at 0c55046 Check for w64-windows machine in Makefile.
(info) https://github.com/FNA-XNA/MojoShader already cloned in MojoShader
(info) https://github.com/KhronosGroup/Vulkan-Headers already cloned in Vulkan-Headers
HEAD is now at c9037d9 spirv: Work around OpLogicalNot being unsupported for ints
HEAD is now at 85470b3 Update for Vulkan-Docs 1.2.165
(info) applying patches to FNA
(info) patch already applied
(info) patch already applied
(info) recursively copying celeste game directory
(info) decompiling the game
(info) patching the game
(info) applying steam-specific patches
(info) building the patched game
Determining projects to restore...
Restored /Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj (in 1.84 sec).
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Draw.cs(302,226): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Draw.cs(318,226): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Draw.cs(334,228): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(27,27): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(29,28): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(31,28): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(33,32): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(35,30): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(37,33): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(39,32): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(41,32): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(43,39): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(45,27): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(47,34): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(49,30): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(51,34): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(53,37): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Monocle/Calc.cs(55,36): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste/RotatingPlatform.cs(9,36): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste/MoveBlock.cs(222,35): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste/MoveBlock.cs(224,33): error CS0103: The name 'MathF' does not exist in the current context [/Users/logan/Desktop/celesteios/celeste-ios/_celeste_decomp/Celeste.csproj]
(error) failed to build the game!`

bro can i have ipa???????
lol

that'd essentially be piracy which is why i wont do that - I'll look into the build error sometime though when i get the motivation

Yeah lol didn’t think about that. Thanks for helping me with this.

silly idea that i just noticed (doubt it'll help but still): try to not run the build script with sudo, it doesn't need elevated permissions and is supposed to work fine without them (also sudo does change environment variables so that might cause issues too)

copying Celeste.exe is intentional, since that is (afaik) always the file extension for executables in dotnet - dll being for libraries. however, you can still load .exe's as libraries, which is also what this project does.
ill look into the rest when i have my iPad back from repair, but thank you for your findings ^^

thats a lot to digest without being able to test anything, thanks for your research !!
im still waiting for my iPad to be out of repair, ill check this out when i have that back

Came back to this and got it working successfully with the help of this PR (https://github.com/RoootTheFox/celeste-ios/pull/14)
my best guess as to why this was occurring is that I was using the Mac version of the game instead of the Linux one as shown in the guide

seems like an issue with the macos build of celeste. installed the game on my linux machine and then copied it over to my mac and that solved the MathF does not exist error for me

I fixed the msbuild not found issue with this command to add it to path:
path+=('/Library/Frameworks/Mono.framework/Versions/Current/bin')

Only issue I'm running into is when applying a patch to FMod (the IPA build fine, but no audio). I'll open another issue for that.


Revert "Use round when setting new chart size"

Reverts chartjs/Chart.js#7221

Accidentally merged #7221 so this reverts that


Fix integration tests withdrawn CVEs

The cleanup of stale GCS-based records changed the CVE record landscape.

Although inspecting these CVEs, they should be converting. Yay, another bug found...

Yay, another bug found...

FYI I expect these to return after https://github.com/google/osv.dev/pull/2214 makes its way into Production...


Javascript localization

do you have options like putting translation also in the javascript? ex. trans.messages.Hello?

Hi @aaesis
We are working on something, but have yet to decide on the best way to implement this feature. We will probably go with something that's compatible with VueJS translations.

hi @sildraug
I'm using Laravel-JS-Localization for JS. It use command to make js file. But it reads translation files from the file system.
Do you have any idea how to make it work together with your extension? I did not find any export command in your plugin.

I was looking for something like that, and this package meets all my expectations, it hurts that I do not have support to use it in JS files, please, if you could add support to this functionality

Please, if you would be so kind as to implement this package to work with native JS

+1

+1


Simplify syscall register and bind

Syscall registration and binding are currently disjoint operations that require the callers to do both for each syscall explicitly.
This can lead to mismatch or missing registration/binds and complicates the code.  Another side effect is that it is difficult/impossible given the current API for callers to filter syscalls at the registration stage but allow them at the binding stage.  One example is the effort to deprecate/disallow syscalls by preventing new deployments of programs that rely on a disallowed syscall while continuing to support existing programs that already rely on them.
Now, all syscalls are registered once, and later the bind operation uses a common context to bind all the previously registered syscalls.  Thus, no longer requiring an explicit bind for each syscall and without the complications of keeping two identical lists of syscalls for registration and bind.
The meat of this PR is in src/vm.rs, the rest of the changes are test related

I think we missed a bind_syscall_context_object in the CLI tool. It won't build anymore.


feat: add pnpm

pnpm (https://pnpm.io/) is a popular package manager that has support for advanced configuration via package.json fields. This keeps those deep keys sorted.

@keithamus do you want me to delete that test?


fix syntax error when upgrading model version

Comment the first line of meta_info['config'] to avoid syntax error.
#3122

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


error when running log commands

Description
error when pressing the commands for logging from within the log-pane.
I get the error below when trying to:
log from neogit pane w/L
log from neogit pane w/b
log from neogit pane w/a
l log for current branch work as it should.
Neovim version
NVIM v0.6.0
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by brew@BigSur
Features: +acl +iconv +tui
See ":help feature-compile"
system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "/usr/local/Cellar/neovim/0.6.0/share/nvim"
Operating system and version
macos 11.5
Steps to reproduce
todo..
Expected behavior
No response
Actual behavior
E5108: Error executing lua ...pack/packer/opt/plenary.nvim/lua/plenary/async/async.lua:14: The coroutine failed with this message: ...nvim/site/pack/p
acker/opt/neogit/lua/neogit/lib/util.lua:124: attempt to get length of local 'str' (a nil value)
stack traceback:
        [C]: in function 'error'
        ...pack/packer/opt/plenary.nvim/lua/plenary/async/async.lua:14: in function 'callback_or_next'
        ...pack/packer/opt/plenary.nvim/lua/plenary/async/async.lua:40: in function 'step'
        ...pack/packer/opt/plenary.nvim/lua/plenary/async/async.lua:43: in function 'execute'
        ...pack/packer/opt/plenary.nvim/lua/plenary/async/async.lua:113: in function 'callback'
        ...vim/site/pack/packer/opt/neogit/lua/neogit/lib/popup.lua:214: in function 'cb'
        ...im/site/pack/packer/opt/neogit/lua/neogit/lib/buffer.lua:298: in function <...im/site/pack/packer/opt/neogit/lua/neogit/lib/buffer.lua:297>

        ...ck/packer/opt/neogit/lua/neogit/lib/mappings_manager.lua:9: in function 'invoke'
        [string ":lua"]:1: in main chunk
Press ENTER or type command to continue

Minimal config
I shall do this and report back.

Duplicate https://github.com/TimUntersberger/neogit/issues/258

I'm sorry, my bad I should have researched issues better.


[Event Request] - Database "Sales header" - function UpdateShipToAddressFromSellToAddress

local procedure UpdateShipToAddressFromSellToAddress(FieldNumber: Integer)
begin
    if ("Ship-to Code" = '') and ShipToAddressEqualsOldSellToAddress then
        case FieldNumber of
            FieldNo("Ship-to Address"):
                "Ship-to Address" := "Sell-to Address";
            FieldNo("Ship-to Address 2"):
                "Ship-to Address 2" := "Sell-to Address 2";
            FieldNo("Ship-to City"), FieldNo("Ship-to Post Code"):
                begin
                    "Ship-to City" := "Sell-to City";
                    "Ship-to Post Code" := "Sell-to Post Code";
                    "Ship-to County" := "Sell-to County";
                    "Ship-to Country/Region Code" := "Sell-to Country/Region Code";
                end;
            FieldNo("Ship-to County"):
                "Ship-to County" := "Sell-to County";
            FieldNo("Ship-to Country/Region Code"):
                "Ship-to Country/Region Code" := "Sell-to Country/Region Code";
        end;
end;

In this function we would like to have an event so it will be possible to handle customer address fields.
F.I.
………
FieldNo("Ship-to Country/Region Code"):
"Ship-to Country/Region Code" := "Sell-to Country/Region Code";
ELSE
OnUpdateShipToAddressFromSellToAddress(Rec,FieldNumber);
end;
end;

Hi @pririis . There are already two events (before and after) fits to your requirement, you can use them.
local procedure UpdateShipToAddressFromSellToAddress(FieldNumber: Integer)
var
IsHandled: Boolean;
begin
IsHandled := false;
OnBeforeUpdateShipToAddressFromSellToAddress(Rec, FieldNumber, IsHandled);
if IsHandled then
exit;
    if ("Ship-to Code" = '') and ShipToAddressEqualsOldSellToAddress then
        case FieldNumber of
            FieldNo("Ship-to Address"):
                "Ship-to Address" := "Sell-to Address";
            FieldNo("Ship-to Address 2"):
                "Ship-to Address 2" := "Sell-to Address 2";
            FieldNo("Ship-to City"), FieldNo("Ship-to Post Code"):
                begin
                    "Ship-to City" := "Sell-to City";
                    "Ship-to Post Code" := "Sell-to Post Code";
                    "Ship-to County" := "Sell-to County";
                    "Ship-to Country/Region Code" := "Sell-to Country/Region Code";
                end;
            FieldNo("Ship-to County"):
                "Ship-to County" := "Sell-to County";
            FieldNo("Ship-to Country/Region Code"):
                "Ship-to Country/Region Code" := "Sell-to Country/Region Code";
        end;

    OnAfterUpdateShipToAddressFromSellToAddress(Rec, xRec, FieldNumber);
end;

Hi,
Sorry but i have looked in BC17 and BC16CU04 on premise. In which version do You find this.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks,
will it also be included in future CU's for BC16 ?

@pririis , no. We do events for the latest version which is 17 for now. If you really need this event on the 16 version please reach out   support channel for a hotfix.


fix: Fixed the condition to use the default value

This PR is for correcting #408 .
The conditions for using the default value specified in the tag have been modified as follows.
before
If there is a default value specified by the tag, always insert it.
this fix
Only if the specified value is nil or Zero value, if there is a default value specified by the tag, it is inserted.

@kemokemo can you add tests?

@jbuchbinder Thank you for your approval.
@nelsam Thank you for your comment. I'll try it. Please give me some time.

@nelsam Sorry for delay. Would you please confirm adding the test?

I see it.  Thanks!  I'm going to pull it down locally and do some extra verification.  Something is nagging at the back of my mind about the FieldByName method ... it might have some scenarios where it doesn't work :thinking:
Yeah, I'll have to double check.  I'm a bit busy today, but I'll set reminders so hopefully I will check it out today or tomorrow.

This code is not correct. Insert plan depends on the first row inserted. If that row uses default value all inserts will have default value.
You could use something like:
CASE WHEN isnull(value) OR value = zerovalue THEN default ELSE value END
Current code also uses reflection for each column even when default value is not set.

@Pomyk Thank you so much for your review. I'll try to fix issues. :thumbsup:

Why this is not merged? This worked on v1. This is an obvious degrade.


No Custom App Launcher Tiles

There is no No Custom App Launcher Tiles under Org Settings / Organization Profile as this article states.
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9488caf7-b791-3002-8749-7fdc4b2ec8b3
Version Independent ID: 08e61eb0-7bcd-f449-46ef-17e209138a8e
Content: Add custom tiles to the app launcher - Microsoft 365 admin
Content Source: microsoft-365/admin/manage/customize-the-app-launcher.md
Service: o365-administration
GitHub Login: @twernermsft
Microsoft Alias: twerner

@jgriff500 Thank you for your feedback. I have not been able to locate the Customer app launcher as well.  We are internally checking with the Author to confirm if this feature has been removed. Based on the update I shall update the document.
Thanks
Sri

@twernermsft Please help us in resolving this issue. Thanks

@twernermsft Gentle reminder, please reply. Thanks

@twernermsft  I'm able to locate the "Custom app launcher tiles".

@jgriff500 I'm able to look at the option currently. Could you please check again? If it does not show up, I'd recommend you contact Microsoft support via the need help link on the admin center to check if there is an issue with the tenant.
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @jgriff500 Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


 This is linked to a QR code in the poster presentation.
Tasklist

[x] AJCAI Page
[x] QR code redirects to page
[x] Page has paper, poster, and relevant information
[x] Contact details, abstract, AJCAI info
[x] Copyright notice

Success Criteria
The QR code links to a page on this website, that page has the paper, poster, and relevant information.

Implemented with commit 476c26f0aff644731044248075592e5738a251e7


Remove button | Define styles

Define styles for the remove button in it's following states:

Default
Hover
Active
Disabled

Done!


ValueDefinition support Debug

It would be good to be able to display the structure of the type if debug formatted so that people can see how they might match against the structure.

I.e. have a Debug implementation that shows whether things are a Composite or Named / Unnamed etc. so that from printout of one value you can see how you might construct a match statement several levels deep... for desub when it printed out the value it wasn't clear quite what the structure was if one wanted to match against it.

Ah, I think I wrote debug impls for things in desub to make values more human readable, and then regretted that and ripped them all out in this crate, so things should already print in the way you're asking for I believe!

Oh, actually I say that but there is one custom Debug impl on ValueDef that I missed, so I'll get rid of that too!


Make ConsulConfigurationSource a public class

ConsulConfigurationSource is package private but it is the return value of the public function, ConsulConfigurationSourceBuilder::build. This causes problems when using kotlin b/c kotlin cannot assign any variable to type ConsulConfigurationSource since it is package private. Since ConsulConfigurationSourceBuilder::build returns an instance of ConsulConfigurationSource, we should make ConsulConfigurationSource a public class.

Fixed by https://github.com/cfg4j/cfg4j/pull/198

Thanks for addressing this!


feat: Pass the primary_ipv6 argument to the AWS provider.

Description, motivation, context
The 5.58.0 release of the Terraform AWS Provider now exposes the PrimaryIpv6 launch template network interfaces parameter via this PR. This parameter causes the first globally-unique IPv6 address to become the primary IPv6 address of instances launched using the template. This is necessary for dual-stack EKS clusters, where the ingress controller's IPv6 address needs to propagate to the ALB during target registration.
The current PR adds the primary_ipv6 flag when used as follows:
module "eks" {
  source  = "terraform-aws-modules/eks/aws"

  # ...
  eks_managed_node_group_defaults = {
    # This is only supported by versions of the terraform-aws-provider >= 5.58.0.
    network_interfaces = [{ primary_ipv6 = true }]
  }
}

Breaking Changes
None.
How Has This Been Tested?

[ ] I have updated at least one of the examples/* to demonstrate and validate my change(s)
[x] I have tested and validated these changes using one or more of the provided examples/* projects
[x] I have executed pre-commit run -a on my pull request

% docker run -e "USERID=$(id -u):$(id -g)" -v $(pwd):/lint -w /lint ghcr.io/antonbabenko/pre-commit-terraform:latest run -a
[INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Terraform fmt............................................................Passed
Terraform docs...........................................................Passed
Terraform validate with tflint...........................................Passed
Terraform validate.......................................................Passed
check for merge conflicts................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
mixed line ending........................................................Passed

I am currently using the examples/eks-managed-node-group/eks-bottlerocket.tf in conjunction with the supplied changes (and additional modifications) in a production environment.

This PR is included in version 20.19.0 :tada:


It can't play music or video in backend and  lock Screen ?

Why not use the HTML5 video element in webview。
I want to make a music app, how can I use react-native to do ?
Thanks.

Hi there! See this issue: https://github.com/brentvatne/react-native-video/issues/152


Add the fixNFTokenTrustlineSurprise amendment:

High Level Overview of Change
Introduces the fixNFTokenTrustlineSurprise  amendment which fixes bugs in interactions between NFTokenOffers and trust lines.
Resolves #4925.
Resolves #4941.
Context of Change
#4925
This issue describes a wide timing window that causes a rare bug.  To understand the process you need some background in how NFTokens work.
When an NFToken is minted, it may have a transfer fee associated with it.  If that transfer fee is non-zero, then a portion of the funds paid for the NFToken's transfer are syphoned off for the NFT's issuer.  That transfer fee is always paid in the same funds as were used for the transfer.
The NFTokenCreateOffer transaction verifies that,

If the NFToken has a transfer fee and
The funds for the transfer are an IOU (not XRP) then
The NFToken issuer must have a trust line for the funds.

If the issuer does not have the necessary trust line then the NFTokenCreateOffer fails.
But here's the timing window.  When the NFTokenCreateOffer happens no funds are transferred.  The funds are transferred when the corresponding NFTokenAcceptOffer transaction completes.  But NFTokenAcceptOffer does not verify the presence of the trust line.
With that background, here are the steps for the bug:

Alice creates a trust line for USD.
Alice mints an NFToken with a non-zero transfer fee.
Becky acquires the NFToken from Alice.
Becky creates an offer to sell the NFToken for USD(100).
Alice removes her trust line for USD.
Carol accepts Becky's offer for the NFToken and pays in USD.
Alice's trust line is recreated and receives the transfer fee in USD.

The bug appears in step 7, where the trust line that Alice deleted in step 5 is recreated.
The fix is simply to put a check in the NFTokenAcceptOffer transaction that is similar to the check in NFTokenCreateOffer.  But the check is transaction changing, so it must be guarded by an amendment.
#4941
This issue describes an even rarer, but related bug.
The checks described in the previous issue involve looking for a trust line connected to the NFToken issuer.  That almost always works.  But there's one situation where it doesn't.  An IOU issuer can always accept any IOU that they issue.  And there is no trust line to represent that capability.  In fact, having a trust line from an account to itself is forbidden.
Here are the steps for the bug:

Alice issues USD
Alice mints an NFToken with a non-zero transfer fee.
Becky acquires the NFToken from Alice.
Becky attempts to creates an offer to sell the NFToken for Alice's USD(100).  This attempt fails with tecNO_LINE

The bug appears in step 4, where Beck cannot create the offer.
The fix is simply to check for the presence of the trust line or if the IOU's issuer is the same as the NFToken issuer.  But the check is transaction changing, so it must be guarded by an amendment.
Type of Change

[x] Bug fix (non-breaking change which fixes an issue)
[x] Tests (unit tests for these changes were added)

Consideration
This NFT-related amendment arrives at about the same time as a different NFT-related amendment: https://github.com/XRPLF/rippled/pull/4945
Should these two pull requests be merged into a single amendment?

Codecov Report
Attention: Patch coverage is 78.94737% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 61.66%. Comparing base (c28e005) to head (b40d3fc).

Files
Patch %
Lines

src/ripple/app/tx/impl/NFTokenAcceptOffer.cpp
78.57%
1 Missing and 2 partials :warning:

src/ripple/app/tx/impl/NFTokenCreateOffer.cpp
80.00%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #4946      +/-   ##
===========================================
+ Coverage    61.64%   61.66%   +0.02%
===========================================
  Files          806      806
  Lines        70967    70986      +19
  Branches     36690    36704      +14
===========================================
+ Hits         43745    43771      +26
+ Misses       19865    19855      -10
- Partials      7357     7360       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

I thought the recreation of the trustline was a feature and that was the reason for disallowIncomingTrustline?

Hi @dangell7, in this particular case the recreation of the trust line is a bug.  The NFToken in the bug report does not have the nft::flagCreateTrustLines flag set.  So the code should not be spontaneously adding trust lines to the issuer of the NFToken.
The test code that I added may be confusing.  Two NFTokens are created:

One with tfTrustLine and
One without.

The NFToken created with tfTrustLine is there to verify that my changes don't break the workflow when that trust line is supposed to be created.  The one without tfTrustLine tests the reported bug.
NFTokens have a lot of (too many?) options.  So it's easy to get tangled up.
Does that make sense?  Feel free to ask further questions.

Hi @dangell7, in this particular case the recreation of the trust line is a bug. The NFToken in the bug report does not have the nft::flagCreateTrustLines flag set. So the code should not be spontaneously adding trust lines to the issuer of the NFToken.
The test code that I added may be confusing. Two NFTokens are created:

One with tfTrustLine and
One without.

The NFToken created with tfTrustLine is there to verify that my changes don't break the workflow when that trust line is supposed to be created. The one without tfTrustLine tests the reported bug.
NFTokens have a lot of (too many?) options. So it's easy to get tangled up.
Does that make sense? Feel free to ask further questions.

Yepp I totally forgot about the flag on the NFToken.

The XLS-25d: NFTokenMintOffer commit ( https://github.com/XRPLF/rippled/commit/9f7c619e4fbbb0e110dc3fa6e9d13dad74b5830f) interacted a great deal with this pull request.

That commit absorbed the fix for issue #4941 that was previously in this pull request.
Similarly, that commit absorbed the unit tests for issue #4941 that were previously in this commit.

So I rebased this commit to the tip of develop and resolved the differences.  I also squashed the commits and changed the commit message to ...

Reflect the changed amendment name and
That this commit only addresses issue #4925 now.

I'm not marking this as passed yet, since I want to allow my reviewers an opportunity to check my work with the squash and rebase. But as far as I'm concerned I think this branch is ready.


Update sts to 2.11.12

Updates software.amazon.awssdk:sts from 2.10.91 to 2.11.12.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "software.amazon.awssdk", artifactId = "sts" } ]

labels: library-update, semver-minor

Codecov Report

Merging #263 into master will not change coverage by %.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #263   +/-   ##
=======================================
  Coverage   47.68%   47.68%
=======================================
  Files          24       24
  Lines         281      281
  Branches        1        1
=======================================
  Hits          134      134
  Misses        147      147

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bdfe9a8...cedbe66. Read the comment docs.


add asio library as header-only alternative to boost::asio

asio library for ubuntu, debian, fedora, arch

Note that (at least for apt) installling asio results in installing and using boost by default. It bit us recently https://github.com/ros2/ci/issues/61


data sets available?

Great job on the scib pipeline. I imagine it was a big headache getting all of these methods to work. I was wondering if the benchmark data set h5ad files were available for download? They seem to be hard coded in config/reproduce_paper.yaml  Thanks!

Hi @zia1138,
The data are all on the figshare here: https://figshare.com/articles/dataset/Benchmarking_atlas-level_data_integration_in_single-cell_genomics_-integration_task_datasets_Immune_and_pancreas/12420968
This is in the revised manuscript... that should be out soon I hope :). We should probably also add this to the README... @mumichae could you add the link?

No hurry on my end.  Thank you again for the great work. It's nice to see a careful evaluation of all of these approaches.

Thanks :).


Quickfix missing import fastpaquet in parquet.py

When trying to run df.to_parquet('dask_dataframe.parquet') from a clean install via pip install dask[dataframe]. It reports
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-16-d56cfa626383> in <module>()
      7
      8 os.chdir('/home/atom/alp/code/scripts')
----> 9 df_with_lakes.to_parquet('lasers_in_lakes.parquet')

~/miniconda3/lib/python3.6/site-packages/dask/dataframe/core.py in to_parquet(self, path, *args, **kwargs)
    955         """ See dd.to_parquet docstring for more information """
    956         from .io import to_parquet
--> 957         return to_parquet(path, self, *args, **kwargs)
    958
    959     def to_csv(self, filename, **kwargs):

~/miniconda3/lib/python3.6/site-packages/dask/dataframe/io/parquet.py in to_parquet(path, df, compression, write_index, has_nulls, fixed_text, object_encoding, storage_options, append, ignore_divisions, partition_on, compute, times)
    367     read_parquet: Read parquet data to dask.dataframe
    368     """
--> 369     import fastparquet
    370     partition_on = partition_on or []
    371

ModuleNotFoundError: No module named 'fastparquet'

So I did pip install fastparquet, and then it reports:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-17-d56cfa626383> in <module>()
      7
      8 os.chdir('/home/atom/alp/code/scripts')
----> 9 df_with_lakes.to_parquet('lasers_in_lakes.parquet')

~/miniconda3/lib/python3.6/site-packages/dask/dataframe/core.py in to_parquet(self, path, *args, **kwargs)
    955         """ See dd.to_parquet docstring for more information """
    956         from .io import to_parquet
--> 957         return to_parquet(path, self, *args, **kwargs)
    958
    959     def to_csv(self, filename, **kwargs):

~/miniconda3/lib/python3.6/site-packages/dask/dataframe/io/parquet.py in to_parquet(path, df, compression, write_index, has_nulls, fixed_text, object_encoding, storage_options, append, ignore_divisions, partition_on, compute, times)
    451     if compute:
    452         writes = delayed(writes).compute()
--> 453         _write_metadata(writes, filenames, fmd, path, metadata_fn, myopen, sep)
    454     else:
    455         out = delayed(_write_metadata)(writes, filenames, fmd, path, metadata_fn,

~/miniconda3/lib/python3.6/site-packages/dask/dataframe/io/parquet.py in _write_metadata(writes, filenames, fmd, path, metadata_fn, myopen, sep)
    475                 fmd.row_groups.append(rg)
    476
--> 477     fastparquet.writer.write_common_metadata(metadata_fn, fmd, open_with=myopen,
    478                                              no_row_groups=False)
    479

AttributeError: 'bool' object has no attribute 'writer'

I fixed it with a simple one-liner import fastparquet in parquet.py. The 'bool' object probably refers to this section, where fastparquet is returned as False? I feel as though the library imports aren't handled particularly well in this file...

[ ] Tests added / passed
[ ] Passes flake8 dask
[ ] Fully documented, including docs/source/changelog.rst for all changes
and one of the docs/source/*-api.rst files for new API

You are probably right that the imports could be better in this file. The intention is to accommodate either parquet-cpp or fastparquet in the reader, if you see the code at read_parquet which checks for the status of the packages.
However I don't understand how your suggestion fixes the behaviour you saw:

if fastparquet is installed, nothing changes, it was already imported at the start of the module
if fastparquet is not installed, to_parquet will give an ImportError and you should never reach this point; if you did, you would get an ImportError here.

Could it be that you did not restart your session after installing fastparquet, or that some dependencies did not install? The value of the fastparquet variable should be either the imported package or False, if it was not found.

@weiji14 do you have any response to @martindurant 's comment above?

This must have been due to lack of restarting your interpreter after installing. The presence of fastparquet and pyarrow are currently only checked at import time - you'd need to restart your interpreter after installing to use them.
Note that in #2868 I change this to avoid doing imports at startup time. This is nicer so that pyarrow and fastparquet aren't imported unless you need them (faster import times), but also would fix your issue you discovered above.
However, since your issue can be fixed by just restarting python, I'm going to close this for now. Thanks.


Implement a logic to not do anything if the same request was already sent previously

This should have multiple checks:

Check to see what values are sent to be verified
Check whether the same request with the same values were already sent within the last 10 minutes
Return an appropriate response saying a similar request was already submitted in the last 10 minutes

/create endpoint is returning the following:
{"meta": {"code": 500, "message": "Unknown Error", "description": "A similar request was already submitted in the last 10 minutes"}}
Please update the message to be either a warning or a failure e.g.
"Same request can't be submitted" OR
"Request already in process" (Recommended) OR
"Duplicate request failed"


ux: Improve endpoint validation by checking type on start

Summary
This PR improves user experience by validating endpoint type on start and including the endpoint display name in the error, which should allow the user to find the culprit more easily.
Checklist

[X] Tested and/or added tests to validate that the changes work as intended, if applicable.
[ ] Added the documentation in README.md, if applicable.

Codecov Report

Merging #323 (5dcbe24) into master (daf8e3a) will decrease coverage by 0.10%.
The diff coverage is 28.57%.

@@            Coverage Diff             @@
##           master     #323      +/-   ##
==========================================
- Coverage   83.02%   82.91%   -0.11%
==========================================
  Files          54       54
  Lines        3859     3864       +5
==========================================
  Hits         3204     3204
- Misses        512      516       +4
- Partials      143      144       +1

Impacted Files
Coverage Δ

core/endpoint.go
75.12% <16.66%> (-1.92%)
:arrow_down:

config/config.go
73.54% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Serializer operator= documentation is misleading

This suggests that serializer is copyable:
https://www.boost.org/doc/libs/develop/libs/beast/doc/html/beast/ref/boost__beast__http__serializer/operator_eq_.html

Needs "(Deleted)" in the javadoc, or we can use #ifndef BEAST_DOXYGEN to remove it from the docs.

http::serializer::operator= is already being rendered as
serializer&
operator=(
    serializer const&) = delete;

Is there anything else we need to do here?
Do we want to remove operator= completely from the table of http::serializer member functions?

Looks good


Add files via upload

Harris academy east Dulwich sixth form

@MahadQorane First stage of verification passed, official domain is harrisdulwichboys.org.uk as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://www.harrisdulwichboys.org.uk/1478/east-dulwich-campus

@MahadQorane First stage of verification passed, official domain is harrisdulwichboys.org.uk as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://www.harrisdulwichboys.org.uk/1478/east-dulwich-campus

@philipto Correction domain should be harrissixthform.org.uk. Thank you for your quick response

@MahadQorane First stage of verification passed, official domain is harrisdulwichboys.org.uk as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://www.harrisdulwichboys.org.uk/1478/east-dulwich-campus

@philipto Correction domain should be harrissixthform.org.uk. Thank you for your quick response

following is the official domain url https://www.harrissixthform.org.uk/

@MahadQorane Thank you for your patience while waiting for a review and processing your request. Pull request merged.


[FEATURE] Spark Take 1 row without sorting optimization

Is your feature request related to a problem? Please describe.
Look at here
If taking just one row with our sorting, we may use GROUP BY and FIRST to solve this problem, it can be a lot faster. Let's add this special handling.

@kvnkho @goodwanghan is this issue taken? Would love to jump on this. Any pointers would be awesome

Hi @meehawk , good to see you in the Fugue repo this time! Thanks for the interest. I still remember you from the Prefect repo. This issue is not taken and you can take a stab at it. Here is some initial guidance.
There is a WorkflowDataFrame class over here. So this is the DAG that Fugue creates to express a computation workflow. The DAG then pushes down the methods to the underlying execution engine (Pandas, Spark, or Dask). So in the same file, you will find a take() method. This calls the Take Processor. This is a generic abstraction for Pandas take, Spark take, and Dask take. So notice the last line calls the take of the underlying execution engine.
You can find the implementations of take attached to each execution engine.

Pandas - https://github.com/fugue-project/fugue/blob/838fdaa794c62e8bdc7f1474818d9491d5d39ed7/fugue/execution/native_execution_engine.py#L320-L354
Spark - https://github.com/fugue-project/fugue/blob/838fdaa794c62e8bdc7f1474818d9491d5d39ed7/fugue_spark/execution_engine.py#L471-L530
Dask - https://github.com/fugue-project/fugue/blob/838fdaa794c62e8bdc7f1474818d9491d5d39ed7/fugue_dask/execution_engine.py#L375-L429

So this specific issue is an optimization for the Spark code. But to know the issue, you need to know the take behavior. Fugue has a unified test suite for all the backends so that users get the same behavior for whatever backend they choose. You can find the tests for take here and it also shows you how to run it. The engine there is the execution engine. For example, for Spark it's this class.
The important thing to know for take as you can see in the tests is that if no partition is specified, it just returns n rows. If a partition is specified, it returns n rows for EACH partition.
This issue proposes special handling to use the native Spark groupby().first() if n is 1 because this will avoid the need to sort each group. An example of this function can be found here. It is relatively newer (PySpark 3.1). This would avoid the sorting logic this take method currently uses. I think you also need to check if the partition has a presort, and if it's ascending use first, and if descending, use last. You can read more about Fugue's partitions here if you are interested, but the gist is you partition by key, and also specify a presort if wanted.
This may be a lot to digest, feel free to ping us in the Slack channel at slack.fugue.ai also and we can discuss this more.


Expand dropdown on hover

The navbar dropdown should expand on hover. Here is the updated links JSON: https://gist.github.com/QuincyLarson/e5739f334bc72daf7cf77005099c2156

Why are you including a gist of the links?

@BerkeleyTrue I thought you mentioned you could add these when you went in and created the PR. I can do this myself, though.

I thought you mentioned that you where going to add these :)
In any case, that should be two issues. Adding links to a json list can be an FTO or a "help wanted" tag.