Add hightlight action

Hi. Thank you for making good open source.
I tried using this open source and I thought I should be able to customize it according to the situation where the background color does not change when the touch happens in each cell.
Thank you for your feedback.
Thank you.

Great work! Thank you for doing this.
I have a few more minor comments, but once those are addressed I think this is ready to merge. Nice job! 😄

Do I have to rebase to synchronize with the upstream master branch?

You just need to rebase these branches and then you can go ahead and merge them.

@marlimox  Finished rebasing

Hi. ci It seems that I can not complete the check all day. Is there anything else I need to do?

Hm yeah that’s strange. @ljharb do you know what might be going on here? I’m unable to admin merge this PR - can you merge it for me or give me admin merge permissions for this repo?

Hi
I think that there are some issues related to not merging, so I will share related links.
https://github.com/travis-ci/travis-ci/issues/9618
https://github.com/travis-ci/travis-ci/issues/10204
https://blog.travis-ci.com/2018-09-27-deprecating-github-commit-status-api-for-github-apps-managed-repositories
Thank you

It's fixed; the issue was that the branch build was marked as required, but it's actually the PR build that should be required (most PRs won't even have a branch build).

@ParkGwangBeom you should be able to merge these PRs now. Let me know if you have any issues!

@marlimox I have been waiting for pr to merge. There are no other issues. :)

Ah you can merge the PR yourself whenever the checks are green! I will go ahead and merge this for you now though


Added AzureMonitor V1 task.

Added AzureMonitor V1 task for querying new Azure Monitor Alerts.

@vithati , please add code owner for the task here.


Add visualization for direction of points

Some points have an orientation, which should be visualized.
Best option is self-created canvas as URL for ol.style.Icon and setting the rotation parameter.

This features is at the moment not supported in Openlayers.
See https://github.com/openlayers/openlayers/issues/2234


Use relaxed SealedBlock in txpool API

Describe the feature
the update fn currently expects a specific block type
https://github.com/paradigmxyz/reth/blob/d68d7c8da0f13acfa356284c4ecf54047f964c25/crates/transaction-pool/src/validate/mod.rs#L209-L209
we can either solve this introducing a new associated type on the validator trait, or provide the generics on the function.
the latter would be easier but would not allow for type specific validator impls, but this should likely be fine and we can still change it later.
TODO

make SealedBlock arg generic over header+body

Additional context
No response

🙋🏽♂


[Feature] Add uninject script

Since it's possible to build and inject BD via npm scripts, but not remove/uninstall/uninject it, I thought this might be a nice addition for people with the manual install/developers.
I am not entirely sure if I have implemented it for all platforms correctly, I kind of just looked at how the installer uninstalls BD and tried to translate that logic here. Would be happy about suggestions/feedback.
I'm also not too sure if the naming "uninject" makes sense, but since the  "install" script refers to installing npm packages here and "inject" to actually injecting BD, "uninject" probably makes more sense than "uninstall" I think.

I wanted to switch my branch but sort of messed up... sorry please just ignore the opening and closing from above

Welp... we have since changed injection method so.. this doesn't work haha

Feel free to re-open if this gets updated.


Patch CVE-2019-19126 in glibc

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[x] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/tools/cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json)
[x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[x] All source files have up-to-date hashes in the *.signatures.json files
[x] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[ ] Ready to merge

Summary

Patches CVE-2019-19126 in glibc
Change Log

Take upstream patch for CVE-2019-19126

Does this affect the toolchain?

YES
Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-2019-19126

Test Methodology

Building locally

Do we have a full build done via pipeline for this? glibc touches so much...
Changes look good though.

@dmcilvaney Sure thing, I'll run a full build over the weekend.

Ran the pipeline, all RPMS are built on AMD64. Going to go ahead and merge, since this change is specific to that arch


Problems when training mar model from scratch

Hi TianHong,
I've been trying to train MAR from scratch recently, but I encountered an issue. If I don’t use the pre-trained parameters you provided and train from scratch, the generated images during training (visualized using the online evaluation command) turn out completely black. I only trained for two epochs on the ImageNet dataset since it’s so large, but I feel like the output shouldn’t be entirely black—at least there should be some noise in the images, right?
To reduce the computational cost, I set num_class=1 and trained on just one class of the ImageNet dataset. Even after training for dozens of epochs, the generated images are still completely black. However, if I use your pre-trained parameters (modified due to num_class=1), the training works fine and I get the expected results.
Did you encounter this issue when training? I’m a bit confused about why training from scratch leads to black images. Do you have any suggestions or things I should pay attention to?
Thank you for your answering!

The randomly initialized model will result in NAN value if you use autocast during inference. If you want to see noise instead of black image, just turn off the autocast during inference https://github.com/LTH14/mar/blob/main/engine_mar.py#L155

Since we use ema=0.9999 by default, you would need at least 100k iterations to get a reasonable result. If you want to see it early, you can disable the ema here https://github.com/LTH14/mar/blob/main/engine_mar.py#L124


⚠ ClamAV has degraded performance

In ad9ef78, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 65 ms

Resolved: ClamAV performance has improved in 578cb8d.


Use static tag for resources in versions, add  for partials

WIP

Coverage decreased (-4.4%) to 88.716% when pulling 912b9a4844dd35cad7d4301b7fa903ce49ddea29 on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage decreased (-4.4%) to 88.716% when pulling 912b9a4844dd35cad7d4301b7fa903ce49ddea29 on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage decreased (-4.4%) to 88.716% when pulling 34e0f019189cc9b0315b2592941c7ed147e0ffb3 on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage remained the same at 93.128% when pulling e6d368bfbc6bcefaebac15cf55e7d88caa76a4e7 on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage increased (+0.002%) to 93.13% when pulling 490dd55b6d2695f36ae520b222945b6763d2dbc6 on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage increased (+0.005%) to 93.133% when pulling 9cf7090da4f1baf2675eb7203660adfea4adeb48 on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage increased (+0.01%) to 93.139% when pulling 8132f2ea8a87fa2d6bf4a390951374312d212c4a on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage increased (+0.01%) to 93.139% when pulling 134f2f31363879aa8e5468f0f41df8102f77b42d on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage increased (+0.01%) to 93.139% when pulling f77cfd39812fc79e47c06661c9143c963b9c960b on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.

Coverage increased (+0.01%) to 93.139% when pulling f77cfd39812fc79e47c06661c9143c963b9c960b on angular-static into e6c97944c1e6e6f52e731a675f7e22818003252e on master.


Add streamer and csrmanager gen

In this RP, we add the streamer and csrManager source code and relevant generation python file as well as format check and unit test for the Scala codes in ci.
These are already working in the https://github.com/KULeuven-MICAS/snax-streamer repo! They are just direct copy-pastes of those repos since we agreed to merge them in here.
The unit tests are also being tested in CI.
This is the first step before making the wrappers!

Wait!, can we maybe first check out the outstanding issues and PRs of that repo?

Also, is util the correct place for this? This feels like it belongs in hw? @rgantonio Have you thought about the final folder tree should look like in the ideal case?

@rgantonio

We verified offline and there is sufficient chisel testing.

What does this mean?

@JosseVanDelm

What does this mean?

I meant Xiaoling and I double-checked locally if it works. Which also appears in the CI that she just made. So double checking, offline and CI. Just to be extra sure there was nothing wrong with what she added.

The csr manager code has specific code for streamer, and it is located under the streamer folder? The CSR manager is a seperate module no?
Can we maybe split this up into different things, because this PR is very hectic:

Add chisel utilities
Add CSR Manager
Add Streamers

The csr manager code has specific code for streamer, and it is located under the streamer folder? The CSR manager is a seperate module no?
Can we maybe split this up into different things, because this PR is very hectic:

Add chisel utilities
Add CSR Manager
Add Streamers

What do you mean by chisel utilities? I think separating CSR manager and streamer in separate directories is fine.

I guess there aren't that much chisel utilities, I thought there were more.
However, I still feel like it belongs in the HW folder. The chisel files are not generated, right? They are the source code of the hardware.

I also think the util folder is a very weird place to put your non-generated files?
Right now the only thing that is included in /util/clustergen is the script to combine multiple files.
The files themselves, like https://github.com/KULeuven-MICAS/snitch_cluster/blob/main/hw/snitch_cluster/src/snitch_cluster_wrapper.sv.tpl are still in the hw folder?
In my opinion there are two options:
We put it in the top-level under /chisel or under /hw/chisel

You might also wonder why in the generated rather than the hw? Well... (1) it is indeed generated noh? (2) we'd like to keep all "hard" source codes in the hw directory, and (3) when we do make clean we can clean easily because at least the generated directory is wiped every make clean. If we put it in hw we have to iterate through every part.

Removing the source generator files makes no sense to me?
(1) The files are not generated, you have handwritten these files, and you use these files to generate other files.
You are never supposed to track generated code inside a git repository.
(2) hw/snitch_cluster/src/snitch_cluster_wrapper.sv.tpl is also not "hard" and is also under hw?
(3) I don't think this is correct; aren't these generated templated files copied to  target/snitch_cluster/build anyway? That is usually the place you want to make clean them? No?

@JosseVanDelm

The files themselves, like https://github.com/KULeuven-MICAS/snitch_cluster/blob/main/hw/snitch_cluster/src/snitch_cluster_wrapper.sv.tpl are still in the hw folder?

Yes, we agreed before not to touch too many things on how ETHZ did it. Since the original plan before was to upstream it into their repository so most of the things that stayed we maintained still. "less changes" as we mentioned long long ago.

Removing the source generator files makes no sense to me?
We're not removing source generator files. I think I mis-explained but rather we do a clean on generated files. In the makefile everytime we do a make clean it cleans the target/snitch_cluster/generated directory. This is where the generated files are dumped. This was in the original Snitch flow. So we thought of putting generated files here, then when we do a clean, we clean it right away.

The files are not generated, you have handwritten these files, and you use these files to generate other files.
You are never supposed to track generated code inside a git repository.

Yes! I agree, don't worry. We're not adding to the track the generated files. I'm just saying when we start generating files, there needs to be a place where to dump the generated files. So we thought of instead of having to create another directory where to put generated files, we figures to re-use the target/snitch_cluster/generated directory from the original flow. I hope that makes sense 😅

hw/snitch_cluster/src/snitch_cluster_wrapper.sv.tpl is also not "hard" and is also under hw?

No, I would say it's "hard" because technically it's like a fixed wrapper. I would "classify" them as hard also only because the signals are "fixed". Just parametrized it with some Python script.  So imagine, it's a source file that the clustergen uses but the structure is the same just things like module names, signal names, or signal bit-widths change. I was thinking all template files should be the same since they are "hard". So people can actually also trace where the signals go to.

I don't think this is correct; aren't these generated templated files copied to target/snitch_cluster/build anyway? That is usually the place you want to make clean them? No?

No, the generated go into the target/snitch_cluster/generated directory. Check out:
https://github.com/KULeuven-MICAS/snitch_cluster/blob/17a98603f84e73115cf9b8d6e07d08357064f46f/target/snitch_cluster/Makefile#L64-L70

As a person who's looking for the source code of the streamer hardware, why would I ever look under util?
Don't burn us for this 😢 but we originally thought to keep the "generators" in one folder only. We actually thought of putting it in its own directory but we also remembered that the clustergen already has its own place. So we contemplated whether the chisel stuff needs to have its own or it needs to be part of the other generators. We decided to put it together with other generators because chisel is a generator thing right? That was the thought process we had. Util as a "tool" for generating or other things. Althought it does seem conflicting because in a way chisel is also a "hardware description" in a way...

How about this, this is the original directory structure proposal we had:
# Put all hard codes into this. For example, hard RTL codes go in here. For example:
hw/
-- hw/snax_accs/
---- hw/snax_accs/acc_1/*.sv
---- hw/snax_accs/acc_2/*.sv
-- hw/snax_wrappers/*.tpl

# Put all chisel stuff in here because technically chisel is a generator which is a utility tool
# Put wrapper gens in here too
util/
-- util/chiselgen/
-- util/wrappergen/

# To be consistent with original Snitch flow, put all generated files in the generated directory
# So it would look something like the one below
target/
-- target/snitch_cluster/
---- target/snitch_cluster/generated/
------ target/snitch_cluster/generated/snax_acc_1/streamer_wrapper.sv
------ target/snitch_cluster/generated/snax_acc_1/snax_acc_1_wrapper.sv

Then since I think I would agree that chisel is an HDL thing then we transform it into:
# Put all hard codes into this. For example, hard RTL codes go in here.
# Chisel is technically "HDL" so let's put it here too
hw/
-- hw/chisel/
---- hw/chisel/project/
---- hw/chisel/src/
---- hw/chisel/*
-- hw/snax_accs/
---- hw/snax_accs/acc_1/*.sv
---- hw/snax_accs/acc_2/*.sv
-- hw/snax_wrappers/*.tpl

# To be consistent with original Snitch flow, put all generated files in the generated directory
# So it would look something like the one below
target/
-- target/snitch_cluster/
---- target/snitch_cluster/generated/
------ target/snitch_cluster/generated/snax_acc_1/streamer_wrapper.sv
------ target/snitch_cluster/generated/snax_acc_1/snax_acc_1_wrapper.sv

I much better like this, thank you!
Diving a bit deeper, how do we organize separate chisel projects?
Maybe we can follow the example from the Chisel book?

I guess our package would be snax and then have subpackages csr_manager, streamer, reshuffler, ...
An accelerator written in scala would then just import the snax package and use the components for connecting everything. What do you think about this?

@jorendumoulin
Isn't @xiaoling-yi sort of following the same structure?

Also, in terms of project, what is one project? I was thinking the entire project directory was the SNAX chisel itself. Then the source files would contain the csr_manager, streamer, then later on the gemm and simd? Or did you mean that each project is like 1 for csr_manager, 1 for streamer , 1 for gemm then 1 for simd?

Yes, I agree, SNAX should be one project, but the problem right now is that CSR manager files and Streamer files are all in the same directory. Therefore maybe we can use subpackages to split them up?
So like this:
src/main/scala/snax/csr_manager
src/main/scala/snax/streamer

Right now the csr manager files are in the streamer folder, which is a bit confusing.

Ah ah ah yes yes, she's doing that right now 😃

Yes, I agree, SNAX should be one project, but the problem right now is that CSR manager files and Streamer files are all in the same directory. Therefore maybe we can use subpackages to split them up?
So like this:
src/main/scala/snax/csr_manager
src/main/scala/snax/streamer

Right now the csr manager files are in the streamer folder, which is a bit confusing.

Don't worry. I will split them explicitly but just some time is needed...

@xiaoling-yi can you please address my previous comments before I rereview?
Also are @jorendumoulin's changes in https://github.com/KULeuven-MICAS/snax-streamer/pull/27 and https://github.com/KULeuven-MICAS/snax-streamer/pull/28 already integrated?
Thanks!

Also are @jorendumoulin's changes in KULeuven-MICAS/snax-streamer#27 and KULeuven-MICAS/snax-streamer#28 already integrated?

yes, they are merged!

Can we please push this through already? I need it for the wrapper part and most comments or suggestions so far are just organizing stuff. Not really design-specific changes. The goal of the PR is just to transfer the CsrManager and Streamer into the repo. I don't want to spend too much time on this. I want to get the tutorial out already. Please?

Yes, we are spending too much time on minor management thing and the development process is severely blocked by these small organizing stuff. We actually have much more important things to do. Shall we just please push this through?

Alright. I've done my best to review this PR as soon as possible and respond to everything quickly, but my questions remain unanswered. I'm disappointed as I thought we were finally going to clean things up a bit, which is in my opinion a necessity for a good tutorial session. I won't block this any further, even though I still feel the remaining issues need to be addressed.

Oks! Your review is helpful! Thanks! I just think it will be easier to fix or clean things when the entire thing is present. We'll keep them in mind of course.

Thanks! All the nice comments are addressed. We leave some comments that we don't think make sense after detailed thought. Thanks for your reviewing effort, nice suggestions, and understanding!


run_upgrades.php for 1.0.0 to 1.0.1 fails

When running the run_upgrades.php script to upgrade to 1.0.1 the query fails:

Starting database migration to version 1.0.1
Preparing: ALTER TABLE dbname.coin_addresses ADD ap_threshold float DEFAULT '0'
failed: Table 'dbname.dbname.coin_addresses' doesn't exist

Looks like the shared DB name is prefixt twice.
Can you try setting the accounts DB name to an empty string?

ACtually, try this first:
git checkout v1.0.1
./upgrade/run_upgrades.php
git checkout v1.0.2
./upgrade/run_upgrades.php

Okay.  Thanks for looking into it.  I'll wait for a fix.

Released 1.0.3, please try that one :-)

Thanks.  That worked!

Thanks for the heads up, somehow I missed this.
It was caused by putting the table in in "`" (back)quotes. It prefixed the database name automatically so it was added twice (once via prefix, once by SQL).


Migrate to Eigen's sparse vector class?

It would be nice to use Eigen's sparse vector class because it supports linear algebra operations without additional work on our side (as opposed to the std::vector<std::pair<int, float>> we are currently using). But we should first do a small benchmark to compare the two options.

where can I find an example for sparse format input in C++ version?


Initial page load

Sorry if this is a basic question, but I couldn't find an answer, and I'm new to react/redux/router.
When the page loads initially, this module doesn't seem to dispatch any redux action. I have a URL, say /user/add, which displays a form to add a new user. When this URL is hit, I want to create a dummy user object with default values in the state, for the purposes of displaying the form correctly and holding it's data temporarily.
The above works fine if I navigate to the URL from other parts of my app, and the LOCATION_CHANGE action is dispatched correctly. However, if this URL was the first page loaded in the app, the LOCATION_CHANGE action isn't fired, which prevents me from setting up the dummy state, causing my app to break.
Is this is the expected behaviour? If so, what's the recommended workaround for handing my case?
I've tried explicitly calling history.push(location.pathname) on page load, and while that does the trick, it has the effect of appending to history, which is not what I want.
I'm using a static server as a backend (no server rendering), and want to stick with stateless functional components if I can.

Yes, it's the expected behavior.  So, some components that need the location state can get it immediately when the page loaded without waiting for another dispatching.
In your case, can you use componentDidMount in your form to set state?

I could use componentDidMount. Certainly works for now. But I wanted to see if I can do this with only stateless components. Thanks for the answer! :)


Integrate Functionary v2.5 + Refactor Functionary Code

Implemented abstract base classes for FunctionaryChatHandler based on version for better maintenance.
Changed chat_format to functionary instead of functionary-v1, functionary-v2, functionary-v2.5. I changed the old chat_format for the old handler to functionary-old. Hope it is not breaking any code at your side. Anyway, can we remove the old handler function already?
Fully integrate Functionary v2.5 for both streaming and non-streaming.
Added code_interpreter feature for v2 and v2.5. Auto activates when user passes in {"type": "code_interpreter"} in tools. Returns a tool call to a tool called python with the argument being the generated code.
Functionary v1 also supports streaming now.

I have tested this PR and it seems to be working just fine.
I used this models.json:
{
    "host": "<IP_ADDRESS>",
    "port": 8080,
    "models": [
      {
        "model": "functionary-small-v2.5.Q4_0.gguf",
        "model_alias": "functionary-small-v2.5-auto-tokenizer",
        "hf_model_repo_id": "meetkai/functionary-small-v2.5-GGUF",
        "hf_pretrained_model_name_or_path": "meetkai/functionary-small-v2.5-GGUF",
        "chat_format": "functionary",
        "n_gpu_layers": 32,
        "offload_kqv": true,
        "n_ctx": 8192,
        "use_mlock": false
      }
    ]
  }

Start server with CONFIG_FILE=models.json  python3 -m llama_cpp.server then calling with an openai client library, something like this:
BASE_URL = "http://<IP_ADDRESS>:8080/v1"
API_KEY = "sk-xxx"
MODEL_ID = "functionary-small-v2.5-auto-tokenizer"

client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

messages = [
    {"role": "user", "content": "what's the weather like in Hanoi?"}
]
tools = [ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g., San Francisco, CA",
                    }
                },
                "required": ["location"]
            }
        }
    }
]

chat_response = client.chat.completions.create(
    model=model,
    messages=messages,
    tools=tools,
)
print(yaml.dump(chat_response.model_dump(), sort_keys=False, explicit_start=True))

Gives the following response:
---
id: chatcmpl-0a698bb0-d38a-4a77-b2e6-2226e09f71c9
choices:
- finish_reason: tool_calls
  index: 0
  logprobs: null
  message:
    content: null
    role: assistant
    function_call: null
    tool_calls:
    - id: call_ePVfb4Gg0aNbCV41cKc9tkAi
      function:
        arguments: '{"location": "Hanoi"}'
        name: get_current_weather
      type: function
created:<PHONE_NUMBER>
model: functionary-small-v2.5-auto-tokenizer
object: chat.completion
service_tier: null
system_fingerprint: null
usage:
  completion_tokens: 12
  prompt_tokens: 126
  total_tokens: 133

Also I realize some of my comments are on existing code which i realize the intent was not to change to make review easier --  but given how much is changing already it may be worth simplifying where possible.


[osg] Enable osg check on baseline

In #10051, due to the influence of other ports, we temporarily disabled the osg check, and now re-enable the check.

osgearth regression will be fixed in #13332.

Use #13385 instead.


Intune says no : is not a valid script

Uploading the script i Intune generates "The uploaded file is not a falid script"
Any tips ?

it's seems that saving the script with notepad is doing the trick.

Right click on RAW and save-as.


Bug in lambda-iam-user-cleanup.py for lab: security/200_labs/200_automated_iam_user_cleanup

On page https://wellarchitectedlabs.com/security/200_labs/200_automated_iam_user_cleanup/1_lambda_iam_cleanup/  there is a reference to "lambda-iam-user-cleanup.py".   In that file, lines 33 and 41 have identical test which I think is a bug due to copy paste.
Line 41:
elif (minAgeUnusedUsersToDelete > 0 and daysSinceUsed >= minAgeUnusedUsersToDelete):
I think it should be
elif (minAgeUnusedUsersToDelete > 0 and daysSinceUsed >= minAgeUnusedUsersToDisable):
Sorry I could not find this file in github else would have filed it as a PR, please fix it.

Resolved in PR305


DestinationId not unique for Stateful services

What should we add or change to make your life better?
During our service development with Yarp, we realized that the destinationId is replicaId for each destination. However, for stateful services, multiple partitions can have the same primary replicaId, which causes Yarp to return only one destination for one replicaId. We believe that it would solve the problem if we could have destinationId be the concatenation of partitionId and replicaId instead.
Why is this important to you?
Currently we can't use destinationId to filter out the right availableDestinations for stateful services because of the above reason, instead, we need to loop through the services which increases the latency. If the destinationId is unique for each destination in stateful services, it would really make our life much easier!

+1. The original SF integration code made the wrong assumption that ReplicaId's / InstanceId's were globally unique. Looks like they aren't, and the proposal above seems reasonable:

We believe that it would solve the problem if we could have destinationId be the concatenation of partitionId and replicaId instead


404 page not found

Hi, I can't get to install the app, after cloning the repository and building it with make, all I get after executing the binary and going to the URL of the web server is "404 page not found". Any idea of what's happening? Thanks

Hi bro, you can refer API document
https://documenter.getpostman.com/view/12796679/UV5agGNr

Hi bro, you can refer API document https://documenter.getpostman.com/view/12796679/UV5agGNr

Hi, it doesn't work, I cannot find a way to get the WebUI working.

Hi bro, you can refer API document https://documenter.getpostman.com/view/12796679/UV5agGNr

Hi, it doesn't work, I cannot find a way to get the WebUI working.

You can try run in docker, if you want runinng in linux vm, you need install a nginx or other web server

The static assets aren't served by the application. That's why an nginx configuration exists which listens on a different port which can be changed to anything (e.g. 443) https://github.com/cylonchau/firewalld-gateway/blob/aff50c483884cbb78ba1cc2ecb592669f8f17139/uranus.nginx.conf

will update
Best regards

寄件者: joe-at-startupmedia @.>
寄件日期: 2024年8月20日 08:57
收件者: cylonchau/firewalld-gateway @.>
副本: Cylon @.>; Comment @.>
主旨: Re: [cylonchau/firewalld-gateway] 404 page not found (Issue #6)
The static assets aren't served by the application. That's why an nginx configuration exists which listens on a different port which can be changed to anything (e.g. 443) https://github.com/cylonchau/firewalld-gateway/blob/aff50c483884cbb78ba1cc2ecb592669f8f17139/uranus.nginx.conf
—
Reply to this email directly, view it on GitHubhttps://github.com/cylonchau/firewalld-gateway/issues/6#issuecomment-2297766267, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHDSYB3HGUUVN6YAYPIIZ6LZSKH6JAVCNFSM6AAAAABMY27WK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJXG43DMMRWG4.
You are receiving this because you commented.Message ID: @.***>

The static assets aren't served by the application. That's why an nginx configuration exists which listens on a different port which can be changed to anything (e.g. 443) https://github.com/cylonchau/firewalld-gateway/blob/aff50c483884cbb78ba1cc2ecb592669f8f17139/uranus.nginx.conf

Please refer install manual

Please refer install manual

Hi bro, you can refer API document https://documenter.getpostman.com/view/12796679/UV5agGNr

Hi, it doesn't work, I cannot find a way to get the WebUI working.

Please refer install manual

@cylonchau no worries I was able to get the web assets working, I was trying to explain to the OP why the binary doesn't serve the static assets. Personally I had problems establishing communications with DBUS because the latest version doesn't allow tcp communication using the method explained.

@cylonchau no worries I was able to get the web assets working, I was trying to explain to the OP why the binary doesn't serve the static assets. Personally I had problems establishing communications with DBUS because the latest version doesn't allow tcp communication using the method explained. I would open a separate issue but I have since moved on.

I haven't figured out how to embed static files into the binary yet. 😥, TCP need configure dbus enable allow remote connection, default is not allow.

cylonchau give me a few, I'll open another issue explaining the TCP dbus issue.

@cylonchau I'm getting a 404 trying to register a user:
Started process:
I0824 09:45:11.755718   17525 server.go:79] FLAG: --one_output="false"
I0824 09:45:11.755727   17525 server.go:79] FLAG: --skip_headers="false"
I0824 09:45:11.755743   17525 server.go:79] FLAG: --skip_log_headers="false"
I0824 09:45:11.755751   17525 server.go:79] FLAG: --sql-driver="sqlite"
I0824 09:45:11.755761   17525 server.go:79] FLAG: --stderrthreshold="2"
I0824 09:45:11.755771   17525 server.go:79] FLAG: --v="5"
I0824 09:45:11.755778   17525 server.go:79] FLAG: --vmodule=""
I0824 09:45:11.757932   17525 model.go:35] Databases stats is {MaxOpenConnections:100 OpenConnections:1 InUse:0 Idle:1 WaitCount:0 WaitDuration:0s MaxIdleClosed:0 MaxIdleTimeClosed:0 MaxLifetimeClosed:0}
I0824 09:45:11.758777   17525 http.go:27] Listening and serving HTTP on <IP_ADDRESS>:2952
I0824 09:45:11.760529   17525 processor.go:60] Async event processor started, waitting task...

[root@startup-nginx-1 ~]# curl --location 'localhost:2952/ping'
{"code":10000,"msg":"operation succeeded","data":"pong"}
[root@startup-nginx-1 ~]# curl --location 'http://<IP_ADDRESS>:2952/auth/signup' --data '{"username": "admin","password": "111"}'

Any ideas?

nvm, I figured it out: https://github.com/cylonchau/firewalld-gateway/blob/67b2e8092059a545ac5c9e32897ba096e8742434/docs/swagger.yaml#L1924
I was referring to that old postman docs.
updated command to crate a user is:
curl -X POST "http://<IP_ADDRESS>:2952/sso/signup" --data '{"username": "admin1","password": "1112"}'

nvm, I figured it out:
https://github.com/cylonchau/firewalld-gateway/blob/67b2e8092059a545ac5c9e32897ba096e8742434/docs/swagger.yaml#L1924
I was referring to that old postman docs.
updated command to crate a user is:
curl -X POST "http://<IP_ADDRESS>:2952/sso/signup" --data '{"username": "admin1","password": "1112"}'

This feature must be enable Database config, if not, those router will not reigster, this is only a firewalld rest api.
Please refer install manual, you must change config and migarte database.
	if !config.CONFIG.MySQL.IsEmpty() || !config.CONFIG.SQLite.IsEmpty() {
		ssoRouter := &sso.SSO{}
		ssoRouter.RegisterUserAPI(ssoGroup)
		// auth route
		authRouter := &auth.Auth{}
		authRouter.RegisterUserAPI(authAPI)

		usersRouter := &user.User{}
		usersRouter.RegisterUserAPI(userAPI)

		tokenRouter := &Token.Token{}
		tokenRouter.RegisterTokenAPI(tokenAPI)

		tagRouter := &tag.Tag{}
		tagRouter.RegisterTagAPI(tagGroup)

		hostRouter := &host.Host{}
		hostRouter.RegisterHostAPI(hostGroup)

		asyncHostRouter := &host.AsyncHost{}
		asyncHostRouter.RegisterAsyncHostAPI(hostGroup)

		templateRouter := &template.Template{}
		templateRouter.RegisterTemplateAPI(templateGroup)

		auditRouter := &audit.Audit{}
		auditRouter.RegisterAuditAPI(auditAPI)
	}
}

@cylonchau I already ran with --migration. This postman link is outdated: https://documenter.getpostman.com/view/12796679/UV5agGNr#060f80aa-7574-4dd1-b145-36847c784cbd

@cylonchau I already ran with --migration. This postman link is outdated: https://documenter.getpostman.com/view/12796679/UV5agGNr#060f80aa-7574-4dd1-b145-36847c784cbd

yes, API doc is already build in, you can use /swagger/index.html API document.

@cylonchau no worries I was able to get the web assets working, I was trying to explain to the OP why the binary doesn't serve the static assets. Personally I had problems establishing communications with DBUS because the latest version doesn't allow tcp communication using the method explained. I would open a separate issue but I have since moved on.

Latest version is can embed assets in binary file.


[Bug] [SVG] Sunburst chart with decals is buggy in new SVG renderer.

Version
5.3.0
Link to Minimal Reproduction
https://echarts.apache.org/examples/editor.html?c=sunburst-borderRadius
Steps to Reproduce

Open the reproduction link
Switch to SVG renderer
Enable decal
Click the outermost piece and back to the root and then move the mouse.
Watch if some errors will be thrown.

Current Behavior
Some errors throw and render abnormally.

Expected Behavior
No errors and render normally
Environment
- OS: Any
- Browser: Any
- Framework: N.A.

Any additional comments?
It may be related to snabbdom/snabbdom#395.

ping @Ovilia @pissang


internalize scap-compliance-checker

hello,
can somebody help me to internalize 'scap-compliance-checker':
https://community.chocolatey.org/packages/scap-compliance-checker
It seems to be to complex for me :)

Perfect. Thanks !


pool timed out while waiting for an open connection during Tenant creation

I've created a multi-tenant solution where PostgreSQL, version 14.10, serves as the database. The module setup for Askar and tenants is outlined as follows:
askar: new AskarModule({
      ariesAskar,
      multiWalletDatabaseScheme: AskarMultiWalletDatabaseScheme.ProfilePerWallet,
    }),
 tenants: new TenantsModule(),

During a single agent session, I can create precisely 8 tenants. However, after re-initializing the agent, I regain the ability to create another set of 8 tenants.
The logger trace indicates the following error for 9th tenant and subsequent ones.
DEBUG: Creating tenant with label core-tenant
DEBUG: Getting tenant agent for tenant '6f2d8b96-aede-430a-94c2-6ec3fe28a105'
DEBUG: Getting context for session with tenant '6f2d8b96-aede-430a-94c2-6ec3fe28a105'
DEBUG: Acquiring tenant session
DEBUG: Increasing current session count to 1 (max: 100)
DEBUG: Acquired tenant session (1 / 100)
DEBUG: Created tenant agent context for tenant '6f2d8b96-aede-430a-94c2-6ec3fe28a105'
DEBUG: Acquired lock for tenant '6f2d8b96-aede-430a-94c2-6ec3fe28a105' to get context
DEBUG: No agent context has been initialized for tenant '6f2d8b96-aede-430a-94c2-6ec3fe28a105', creating one
INFO: Initializing wallet 'tenant-6f2d8b96-aede-430a-94c2-6ec3fe28a105' {
  "storage": {
    "type": "postgres",
    "config": {
      "host": "localhost:5432",
      "connectTimeout": 10
    },
    "credentials": "[*****]"
  },
  "id": "tenant-6f2d8b96-aede-430a-94c2-6ec3fe28a105",
  "key": "[*****]",
  "keyDerivationMethod": "RAW"
}
DEBUG: Opening wallet for profile 'tenant-6f2d8b96-aede-430a-94c2-6ec3fe28a105'
ERROR: Error opening wallet for profile 'tenant-6f2d8b96-aede-430a-94c2-6ec3fe28a105' {
  "error": {
    "name": "Error",
    "stack": "Error: Backend error\nCaused by: pool timed out while waiting for an open connection\n    at cb (/home/frcs/nordivo/hovi/hovi-afj-agents/agents/node_modules/@hyperledger/aries-askar-nodejs/src/NodeJSAriesAskar.ts:167:25)\n    at Object.<anonymous> (/home/frcs/nordivo/hovi/hovi-afj-agents/agents/node_modules/ffi-napi/lib/callback.js:66:27)",
    "message": "Backend error\nCaused by: pool timed out while waiting for an open connection",
    "code": 1
  },
  "errorMessage": "Backend error\nCaused by: pool timed out while waiting for an open connection"
}
DEBUG: Releasing session because an error occurred while getting the context for tenant 6f2d8b96-aede-430a-94c2-6ec3fe28a105 {
  "errorMessage": "Error opening wallet for profile 'tenant-6f2d8b96-aede-430a-94c2-6ec3fe28a105'"
}

Now, If I use following wallet scheme I can create as many tenants as I want in a single agent session.
askar: new AskarModule({
      ariesAskar,
      multiWalletDatabaseScheme: AskarMultiWalletDatabaseScheme.DatabasePerWallet
    }),

Following is my package.json info.
"dependencies": {
    "@aries-framework/anoncreds": "0.4.2",
    "@aries-framework/anoncreds-rs": "0.4.2",
    "@aries-framework/askar": "0.4.2",
    "@aries-framework/cheqd": "0.4.2",
    "@aries-framework/core": "0.4.2",
    "@aries-framework/indy-sdk": "0.4.2",
    "@aries-framework/indy-vdr": "0.4.2",
    "@aries-framework/node": "0.4.2",
    "@aries-framework/tenants": "0.4.2",
    "@hyperledger/anoncreds-nodejs": "^0.1.0",
    "@hyperledger/aries-askar-nodejs": "^0.1.0",
    "@hyperledger/indy-vdr-nodejs": "^0.1.0",
    "dotenv": "^16.3.1"
  },
  "devDependencies": {
    "ts-node": "^10.4.0",
    "typescript": "^5.2.2",
    "prettier": "^3.1.1"
  },
  "resolutions": {
    "@types/node": "^16.11.7",
    "ref-napi"<EMAIL_ADDRESS>  }

This latest @hyperledger/aries-askar-nodejs": "0.2.0-dev.5 package resolves the issue.


Readme contains 404's

Tell us about the bug
Readme contains a bunch of great tutorials but links are pointing to the wrong place.

Hey Sion!
Thanks a lot for creating the issue and the PR.
It's fixed in https://github.com/quixio/quix-streams/pull/328.
I'll close the https://github.com/quixio/quix-streams/pull/327 (in 328 the links lead to quix.io/docs instead of github)


input task should allow an additional certificate-resource-name input

This fixes an issue where none of the workflow tests were testing generate_certificate step. Additionally, this alters the export task to allow 2 inputs labels now:

The file to export
The certificate of the exported resource (in order to wait for generation). https://github.com/world-federation-of-advertisers/cross-media-measurement/pull/1330/files previously added this step but because those workflows were not being tested, this issue was missed.

This change is


Serialization of atoms with msgpack causes exception

For example:
wamper_protocol:serialize(ok, raw_msgpack).
** exception error: bad argument: ok
Replacing a line https://github.com/bwegh/wamper/blob/master/src/wamper_protocol.erl#L142
with
Enc = case msgpack:pack(Message, [{format,map}, {allow_atom, pack}]) of
seems to resolve the issue.

I do understand your point of view @wcreek , yet do not see that as an issue.
It will keep you from sending messages that might be ment for internal use only, which should be any message still containing atoms (e.g. having payload from your middleware are entrance to the one checking the message before sending it out) .
In the callback erwa_middleware:check_out_message/2 you could alter all the atoms to valid binaries.
If you have an Idea how to easily forward the flag to allow serialization of atoms I will gladfully add it.
thanks for your feedback.


Version 0.1.2 not compatible with WebStorm 2022.3

According to the WebStorm plugin settings, Mongo DB Browser requires IDE build 222.* or earlier. WebStorm 2022.3 is IDE build #WS-223.7571.168 (on Windows).

It should work but will check.  I think JetBrains has again updated API for IDE versions going forward.

Hi @HelgeStenstrom 222.* means versions including 2022.2.*. You can check the https://plugins.jetbrains.com/plugin/20002-mongo-db-browser/versions for more details regarding the support.

Added compatibility for latest IDE versions 223.* in new plugin version 2022.3.2


BEC - Stages 2-3 UEBA updates

Required items, please complete
Change(s):

Added UEBA tables IdentityInfo and BehaviorAnalytics where applicable in all of the BEC scenario stage 2-3 queries.

Reason for Change(s):

Adding additional UEBA data for context.

Version Updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

Testing Completed:

Yes

Checked that the validations are passing and have addressed any issues that are present:

Yes

Hello @jannieli please address to @aprakash13 comments

Hello @jannieli @petebryan has requested changes

Hello @jannieli please remove the branch conflicts

Hello @petebryan & @aprakash13 @jannieli has made some changes

Hello @jannieli any updates on the above

Hello @petebryan, @aprakash13 @jannieli has made some changes

Please update the branch from master as well

Template Id: 99885ff5-00cf-49e8-9452-6de6aba2a5c7 is not valid in Line: 31 col: 8
Errors: The column 'IP_0_Address' must exist on both sides of the join., Code: 'KS145', Severity: 'Error', Location: '1401..1413'
please resolve this issue

Hello @jannieli any updates on above

Hello @jannieli we are waiting for your update, and please also resolve the branch conflicts

Hello @jannieli, Thank you for committing the changes, you can expect an update till 26 Jul 2023


Error: Running without the SUID sandbox!

I'm experiencing an error whilst trying to run this on a Raspberry Pi 3, not sure if this is the same for the 2. The error I'm seeing is as follows:
13.04.16 21:51:10 [+0100] [75:0413/205110:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
13.04.16 21:51:23 [+0100] [75:0413/205123:ERROR:browser_gpu_channel_host_factory.cc(145)] Failed to create channel.

Please let me know if there is any more information you need to debug this.

hi, Those errors shouldn't prevent the execution of your app (i'm investigating them anyways).
How did you set the env vars?

Similar:
[77:0525/040715:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
Can't get the gui to launch and running the direct IP address yields that nothing is running on 80.
Any ideas? Raspberry Pi.

this is because i made the root user start electron. As a template purpose, this is good since people tinkering/learning with GPIO or other interfaces won't get stuck on permission issues

@curcuz what's the suggested method to do this properly?  ;)

@dovy apparently, just the way it is already 😄  https://github.com/electron/electron/issues/1482


Implement Set method for setting and replace raw integers with constants defined in http pacakge

implement Set method of setting
replace integers with http constants

/cc @cbron


Prepare for Modules (vgo support)

This is a placeholder ticket to collect the various ideas and implementation steps currently discussing Golang module support.
A module is a collection of packages, denoted by a common filesystem root, in which in it's simplest form a file named go.mod resides. As a consequence there is no more need of GOPATH.
A proposal and reference implementation are under way.
For the design of go-mode a couple of questions come up like shall a future go-mode support both environments of set GOPATH and pure Golang module setups?

For the design of go-mode a couple of questions come up like shall a future go-mode support both environments of set GOPATH and pure Golang module setups?

Yes, it should.

any update on this?

go-mode works pretty well for me when using modules. Are there concrete things that aren't working for people right now?

Things that come to mind:

go-packages-add won't work in a module environment; but it also doesn't work correctly in most GOPATH environments, anymore. I recommend deprecating the function and recommending people use goimports instead.
we could add a major mode for go.mod files. It should be fairly trivial, needing only a very basic syntax table and simple indentation.
We should check go-mode for code that expects a GOPATH. Some I've found so far:

go-set-project

We've added a new major mode for the go.mod file in 50e15c7f20fb277555e2019e95ff28434bb6dd7a

We've deprecated all GOPATH-related functionality. Building projects using GOPATH is not expected to be supported by Go for much longer. go-packages-add should in theory work fine if setting go-packages-function to go-packages-go-list, but in reality, most people use goimports or gopls, anyway, and don't add imports manually.
go-mode has been working fine with modules for a while, so I think we can close this issue.


Typos in timeseries_anomaly_detection.py

Fixes minor typos: "recontruct" should be "reconstruct"

@fchollet No problem.  Also fixed .md and .ipynb.

@dennisvang Thanks for the PR! You'll need to sign the CLA before we can merge this.


CMake build

I started building a CMake version of the build so that I can move away from gyp. I don't think the current directory structure works since I don't think CMake can handle multiple projects very well. Most googling I saw requires you to create multiple CMake builds.
So let's investigate what is needed to move to CMake.

Finished this a few months ago.


⚠ Bofrost.lu has degraded performance

In 40d2f08, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1067 ms

Resolved: Bofrost.lu performance has improved in dab0d46 after 13 minutes.


app.py needs comment

import os
import importlib
import yaml
import time
import logging
import pytorch_lightning as pl
from torch.utils.data import DataLoader
from pytorch_lightning.loggers import TensorBoardLogger
from pytorch_lightning.strategies import DDPStrategy
from flask import Flask, session, render_template, request, redirect, url_for, send_from_directory
from auth import auth, login_required
from project import project
from dashboard import dashboard
from tasks import tasks
from models import models
from runs import runs
from datasets import dataset
from optimize import optimizations
from monitor import monitor_bp
from deploy import deploy_bp...
there is no comment

we need to find notations


feat(dbm-services): dbha optimize log info

关联研发信息
dev_branch: feat/dbha-log-grace/#6815
target_branch: v1.5.0


file-config.json show all project files & folders ( dist, config,...)

i've tried file-config.json but I don't think the server (localhost:3000) should show all project files/folders

This is working as intended. The urn:solid-server:default:variable:rootFilePath variable defaults to the current working directory; so you can either:

run the command from any other directory
set rootFilePath via the --rootFilePath (or -f) command-line flag

Ok @RubenVerborgh if this is was is intended, no matter for me...
But does it mean that we could that way update / modify the server files on the fly by itself ? 🤔

Yeah, it's a filesystem. Just like you could open your text editor's sources in the editor.
But you probably want to configure that differently.


KYLIN-4250 FechRunnner should skip the job to process other jobs instead of throwing exception when the job section metadata is not found

https://issues.apache.org/jira/browse/KYLIN-4250

Can one of the admins verify this patch?

conflicts， close first


feat: disable metrics by default

By default, we want to only collect audit logs.
Go through all modules and make sure metrics are disabled by default.
Also consider adding a new variable to enable metrics if needed.

Modules where metrics are enabled:

[ ] App Config
[ ] Function App
[ ] Key Vault
[ ] Log Analytics
[ ] Service Bus
[ ] Web App

The mentioned modules have now have metrics disabled by default, and have been made into a variable for easier enabling if needed. Any comments before we can close this isssue @equinor/terraform-baseline ?

@kov117 Any more modules that could have this implemented?

there were quite a few, i will include them to the to the list and continue the work

And with that all the modules have now metrics disabled by default, except for SQL but it has its reasons Ref. and with that i will close this issue 👋🏽.


runtime/cgo: arm5 sigill

What version of Go are you using (go version)?
current-ish master. see the linux-arm-arm5spacemonkey column on https://build.golang.org/
What operating system and processor architecture are you using (go env)?
GOARCH="arm"
GOOS="linux"
GOARM=5
What did you do?
ran all.bash
What did you expect to see?
the tests to pass
What did you see instead?
##### ../misc/cgo/testplugin
SIGILL: illegal instruction
PC=0x1bc0b4 m=0 sigcode=1
signal arrived during cgo execution

goroutine 1 [syscall, locked to thread]:
runtime.cgocall(0x11ce20, 0x10545c54, 0x1050e108)
    /home/builder/stage0scratch/tmp/workdir/go/src/runtime/cgocall.go:132 +0xb8 fp=0x10545c3c sp=0x10545c20
plugin._Cfunc_pluginOpen(0x1db158, 0x1050e108, 0x0)
    plugin/_obj/_cgo_gotypes.go:98 +0x38 fp=0x10545c50 sp=0x10545c3c
plugin.open.func3(0x1db158, 0x1050e108, 0x10510140)
    /home/builder/stage0scratch/tmp/workdir/go/src/plugin/plugin_dlopen.go:61 +0x8c fp=0x10545c6c sp=0x10545c50
plugin.open(0x1422e1, 0xa, 0x0, 0x0, 0x0)
    /home/builder/stage0scratch/tmp/workdir/go/src/plugin/plugin_dlopen.go:61 +0x14c fp=0x10545d9c sp=0x10545c6c
plugin.Open(0x1422e1, 0xa, 0x0, 0x0, 0x0)
    /home/builder/stage0scratch/tmp/workdir/go/src/plugin/plugin.go:30 +0x24 fp=0x10545db4 sp=0x10545d9c
main.main()
    /home/builder/stage0scratch/tmp/workdir/go/misc/cgo/testplugin/src/host/host.go:55 +0x38 fp=0x10545fbc sp=0x10545db4
runtime.main()
    /home/builder/stage0scratch/tmp/workdir/go/src/runtime/proc.go:185 +0x1e0 fp=0x10545fe4 sp=0x10545fbc
runtime.goexit()
    /home/builder/stage0scratch/tmp/workdir/go/src/runtime/asm_arm.s:970 +0x4 fp=0x10545fe4 sp=0x10545fe4

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /home/builder/stage0scratch/tmp/workdir/go/src/runtime/asm_arm.s:970 +0x4

For example,
linux-arm-arm5spacemonkey at d0ff9ece2b36bd4470b4388602a757d50c4c3607
https://build.golang.org/log/70a355740ca968705fdd530ea48828136bad9a7a
Notably, these are the first real ARM5 builders we've ever had. The old "arm5" builders we had were just modern-ish ARMv6-or-7-or-something (Scaleway C1) machines running the ARM5 binaries, which meant we got away with cheating (illegal instructions) without realizing it.
So this is probably some codegen not respecting the $GOARM=="5".
/cc @ianlancetaylor @randall77 @josharian @minux @cherrymui

(That's one example, but the whole build column is like that.)

Oh, looking closer finally, this is cgo/testplugin, and during a cgo call, so the C compiler is not generating the ARM5 code. I guess we might need to pass down special flags to the C compiler to change its target processor version?
@ianlancetaylor?
Is this something that cmd/go needs to do?

There are a few places in the toolchain that pass architecture-specific options to the C compiler:

Builder.gccArchArgs in cmd/go/internal/work/build.go
hostlinkArchArgs in cmd/link/internal/lib.go
Package.gccMachine in cmd/cgo/gcc.go

I think it would be reasonable to modify those places (or unify them!) and arrange that if GOARM=5 we pass -march=armv5.

@zeebo, interested in tackling this?

Sure.

We have an initial patchset ready for this but even though -march=armv5 is being passed, it still doesn't seem to fix the issue:
# ~/go/bin/go env
GOARCH="arm"
GOBIN=""
GOEXE=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOOS="linux"
GOPATH=""
GORACE=""
GOROOT="/root/go"
GOTOOLDIR="/root/go/pkg/tool/linux_arm"
GCCGO="gccgo"
GOARM=""
CC="gcc"
GOGCCFLAGS="-fPIC -marm -march=armv5 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build236399254=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

# ~/go/bin/go tool dist test -run testplugin

##### ../misc/cgo/testplugin
SIGILL: illegal instruction
PC=0x1bb608 m=0 sigcode=1
signal arrived during cgo execution

goroutine 1 [syscall, locked to thread]:
runtime.cgocall(0x11cdd0, 0x10543c54, 0x1050e108)
	/root/go/src/runtime/cgocall.go:132 +0xb8 fp=0x10543c3c sp=0x10543c20
plugin._Cfunc_pluginOpen(0x1da158, 0x1050e108, 0x0)
	plugin/_obj/_cgo_gotypes.go:98 +0x38 fp=0x10543c50 sp=0x10543c3c
plugin.open.func3(0x1da158, 0x1050e108, 0x10512180)
	/root/go/src/plugin/plugin_dlopen.go:61 +0x8c fp=0x10543c6c sp=0x10543c50
plugin.open(0x142241, 0xa, 0x0, 0x0, 0x0)
	/root/go/src/plugin/plugin_dlopen.go:61 +0x14c fp=0x10543d9c sp=0x10543c6c
plugin.Open(0x142241, 0xa, 0x0, 0x0, 0x0)
	/root/go/src/plugin/plugin.go:30 +0x24 fp=0x10543db4 sp=0x10543d9c
main.main()
	/root/go/misc/cgo/testplugin/src/host/host.go:55 +0x38 fp=0x10543fbc sp=0x10543db4
runtime.main()
	/root/go/src/runtime/proc.go:185 +0x1e0 fp=0x10543fe4 sp=0x10543fbc
runtime.goexit()
	/root/go/src/runtime/asm_arm.s:970 +0x4 fp=0x10543fe4 sp=0x10543fe4

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/root/go/src/runtime/asm_arm.s:970 +0x4

trap    0x6
error   0x0
oldmask 0x0
r0      0xb5d67c30
r1      0x1b8ae8
r2      0x1b8ae8
r3      0x0
r4      0xb5d670b8
r5      0x1d599c
r6      0x1d5978
r7      0x12e6e8
r8      0xb5d767b0
r9      0xb5d3c2b4
r10     0xb5d3c468
fp      0xb5d76799
ip      0xb5d11430
sp      0xb5d66f74
lr      0x56599
pc      0x1bb608
cpsr    0x10
fault   0x0
2017/03/24 23:55:46 Failed: exit status 2
2017/03/24 23:55:46 FAILED

Here is the diff (with some stuff elided, like the deps changes):
diff --git a/src/cmd/cgo/gcc.go b/src/cmd/cgo/gcc.go
index a740748..849a6ab 100644
--- a/src/cmd/cgo/gcc.go
+++ b/src/cmd/cgo/gcc.go
@@ -9,6 +9,7 @@ package main
 import (
        "bytes"
+       "cmd/internal/obj"
        "debug/dwarf"
        "debug/elf"
        "debug/macho"
@@ -1202,7 +1203,11 @@ func (p *Package) gccMachine() []string {
        case "386":
                return []string{"-m32"}
        case "arm":
-               return []string{"-marm"} // not thumb
+               args := []string{"-marm"} // not thumb
+               if obj.GOARM == 5 {
+                       args = append(args, "-march=armv5")
+               }
+               return args
        case "s390":
                return []string{"-m31"}
        case "s390x":
diff --git a/src/cmd/go/internal/work/build.go b/src/cmd/go/internal/work/build.go
index c09d8d3..1177141 100644
--- a/src/cmd/go/internal/work/build.go
+++ b/src/cmd/go/internal/work/build.go
@@ -3074,7 +3074,11 @@ func (b *Builder) gccArchArgs() []string {
        case "amd64", "amd64p32":
                return []string{"-m64"}
        case "arm":
-               return []string{"-marm"} // not thumb
+               args := []string{"-marm"} // not thumb
+               if obj.GOARM == 5 {
+                       args = append(args, "-march=armv5")
+               }
+               return args
        case "s390x":
                return []string{"-m64", "-march=z196"}
        case "mips64", "mips64le":
diff --git a/src/cmd/link/internal/ld/lib.go b/src/cmd/link/internal/ld/lib.go
index 7f05682..8cec542 100644
--- a/src/cmd/link/internal/ld/lib.go
+++ b/src/cmd/link/internal/ld/lib.go
@@ -1257,7 +1257,11 @@ func hostlinkArchArgs() []string {
        case sys.AMD64, sys.PPC64, sys.S390X:
                return []string{"-m64"}
        case sys.ARM:
-               return []string{"-marm"}
+               args := []string{"-marm"}
+               if obj.GOARM == 5 {
+                       args = append(args, "-march=armv5")
+               }
+               return args
        case sys.ARM64:
                // nothing needed
        case sys.MIPS64:

What is the instruction at PC value 0x1bb608?

Is this the right thing?
(gdb) disas 0x1bb608
Dump of assembler code for function runtime.finalizer1:
=> 0x001bb608 <+0>:                     ; <UNDEFINED> instruction: 0xf7bdef7b
   0x001bb60c <+4>:     ldrdeq  r0, [r0], -lr
End of assembler dump.

I figured out how to keep the binary/so files around. Would a copy of those be helpful?

Poking around with gdb some more, I can't get the program to finish the topmost stack frame at
#0  _dl_init (main_map=main_map@entry=0x1db1a8, argc=1, argv=0xbefff834, env=0xbefff83c) at dl-init.c:86
#1  0xb6fe3838 in dl_open_worker (a=<optimized out>) at dl-open.c:577
#2  0xb6fdf080 in _dl_catch_error (objname=0xb6fdf080 <_dl_catch_error+120>, objname@entry=0xbefff3ec, errstring=0xb6ff5518, errstring@entry=0xbefff3f0, mallocedp=0xbefff3ec, mallocedp@entry=0xbefff3eb, operate=0xbefff3eb, args=args@entry=0xbefff3f4) at dl-error.c:187
#3  0xb6fe2f00 in _dl_open (file=0x1da158 "/root/go/misc/cgo/testplugin/plugin1.so", mode=-2147483390, caller_dlopen=0x11ce20 <_cgo_e64586f1776f_Cfunc_pluginOpen+80>, nsid=-2, argc=1, argv=0xbefff834, env=0xbefff83c) at dl-open.c:661
#4  0xb6fbcbc0 in dlopen_doit (a=0xbefff640) at dlopen.c:66
#5  0xb6fdf080 in _dl_catch_error (objname=0xb6fdf080 <_dl_catch_error+120>, errstring=0xb6ff5518, mallocedp=0x1db16c, operate=0x1db168, args=0xbefff640) at dl-error.c:187
#6  0xb6fbd30c in _dlerror_run (operate=0xb6fbcb40 <dlopen_doit>, args=args@entry=0xbefff640) at dlerror.c:163
#7  0xb6fbcc90 in __dlopen (file=<optimized out>, mode=mode@entry=258) at dlopen.c:87
#8  0x0011ce20 in pluginOpen (err=0x1050e108, path=<optimized out>) at /root/go/src/plugin/plugin_dlopen.go:19
#9  _cgo_e64586f1776f_Cfunc_pluginOpen (v=0x10545c54) at cgo-gcc-prolog:72
#10 0x000c3d5c in runtime.asmcgocall () at /root/go/src/runtime/asm_arm.s:526
#11 0x00000020 in ?? ()

This seems to be the jump that causes stuff to go wrong (it picks a different address to jump to when I have the breakpoint set. No idea):
(gdb) start
Temporary breakpoint 26 at 0x11b824: file /root/go/misc/cgo/testplugin/src/host/host.go, line 51.
Starting program: /root/go/misc/cgo/testplugin/host
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabi/libthread_db.so.1".
[New Thread 0xb6de9460 (LWP 3071)]
[New Thread 0xb65e9460 (LWP 3072)]

Temporary breakpoint 26, main.main () at /root/go/misc/cgo/testplugin/src/host/host.go:51
51              if got, want := common.X, 3*5; got != want {
(gdb) b *0xb6fe3870
Breakpoint 27 at 0xb6fe3870: file dl-open.c, line 601.
(gdb) c
Continuing.
Breakpoint 27, 0xb6fe3870 in dl_open_worker (a=<optimized out>) at dl-open.c:601
(gdb) info registers
r0             0xb5d97c30<PHONE_NUMBER>
r1             0x1b8ae8 1805032
r2             0x1b8ae8 1805032
r3             0x0      0
r4             0xb5d970b8<PHONE_NUMBER>
r5             0x1d599c 1923484
r6             0x1d5978 1923448
r7             0x12e6e8 1238760
r8             0xb5da67b0<PHONE_NUMBER>
r9             0xb5d6c2b4<PHONE_NUMBER>
r10            0xb5d6c468<PHONE_NUMBER>
r11            0xb5da6799<PHONE_NUMBER>
r12            0xb5d41430<PHONE_NUMBER>
sp             0xb5d96f74       0xb5d96f74
lr             0x1bb5dc 1816028
pc             0xb6fe3870       0xb6fe3870 <dl_open_worker+876>
cpsr           0x40000010<PHONE_NUMBER>
(gdb) disas 0xb6fe3870, 0xb6fe3874
Dump of assembler code from 0xb6fe3870 to 0xb6fe3874:
   0xb6fe3870 <dl_open_worker+876>:     bx      lr
(gdb) si
0x001bb5dc in runtime.lastmoduledatap ()
(gdb) si
0x001bb5e0 in runtime.maxstacksize ()
(gdb)

Program received signal SIGILL, Illegal instruction.
0x001bb5e0 in runtime.maxstacksize ()

Let me know if there's anything useful I can do to help debug this more, because I don't really know what I'm looking at or if any of this is helpful. 😃

@ianlancetaylor, would you like SSH access to the builder? We're happy to get you on there if that will help. If so, get me your SSH pubkey and I'll get an account set up.

Ping? We're stuck on this.

If you are stuck on this because misc/cgo/testplugin is failing on ARM5, then I think the answer is to disable that test.  It's clear that plugin support is very patchy at the moment, and we shouldn't let plugin test failures hold us up anywhere.  (Does the test fail on normal ARM?  I know very little about Go on ARM myself.)
The comment above suggests that the program is somehow trying to execute the code at runtime.finalizer1, but that is a variable (a []byte), not a function.  So the program is crashing because it is trying to execute data as though it were code.

CC @crawshaw because this appears to be plugin related.

On Mar 25, 2017 5:58 PM, "Jeff"<EMAIL_ADDRESS>wrote:
Is this the right thing?
(gdb) disas 0x1bb608
Dump of assembler code for function runtime.finalizer1:
=> 0x001bb608 <+0>:                     ;  instruction: 0xf7bdef7b
0x001bb60c <+4>:     ldrdeq  r0, [r0], -lr
End of assembler dump.
I figured out how to keep the binary/so files around. Would a copy of those
be helpful?
runtime.finalizer1 is generated by our compiler, and it's using armv6 ldrd.
I'd check the SSA rule for that instruction. And what's the first
instruction at 0x1bb608.

@randall77, got any time to check this out?
Looks like it might be the Go compiler's fault and not cmd/go failing to pass down flags to gcc.

runtime.finalizer1 isn't code, it is data.  Disassembling it is sure to give you junk.
If the PC actually reached this global variable, something else is very wrong.

(gdb) info registers
lr             0x1bb5dc 1816028
pc             0xb6fe3870       0xb6fe3870 <dl_open_worker+876>
(gdb) disas 0xb6fe3870, 0xb6fe3874
Dump of assembler code from 0xb6fe3870 to 0xb6fe3874:
   0xb6fe3870 <dl_open_worker+876>:     bx      lr
(gdb) si
0x001bb5dc in runtime.lastmoduledatap ()

runtime.lastmoduledatap is also a data symbol. But the branch did what it was told to, so why was lr set to runtime.lastmoduledatap?

@zeebo, a few questions:

Which version of libc are you using? (So we can match up source lines.)

Can you post the disassembly of all of dl_open_worker? I'm particularly interested in the instructions leading up to the bx lr, but might as well post the whole thing.

Upon entry to dl_open_worker, what are the registers? (In particular lr, but, again, might as well dump them all.)

from dpkg, 2.19-18+deb8u7.

https://gist.github.com/zeebo/9bf6059544521f0b311efe4e6953f312

Breakpoint 1, dl_open_worker (a=0xbefff924) at dl-open.c:197
197	dl-open.c: No such file or directory.
(gdb) info registers
r0             0xbefff924<PHONE_NUMBER>
r1             0x0	0
r2             0x97	151
r3             0xb6fe3504<PHONE_NUMBER>
r4             0x0	0
r5             0x80000102<PHONE_NUMBER>
r6             0xb6fff050<PHONE_NUMBER>
r7             0x1d9158	1937752
r8             0xb6fbcbc0<PHONE_NUMBER>
r9             0x122e30	1191472
r10            0x1c62c8	1860296
r11            0xbefff964<PHONE_NUMBER>
r12            0xbefff7a8<PHONE_NUMBER>
sp             0xbefff750	0xbefff750
lr             0xb6fdf080<PHONE_NUMBER>
pc             0xb6fe3504	0xb6fe3504 <dl_open_worker>
cpsr           0x60000010<PHONE_NUMBER>

I took the liberty of running a disas on the contents of lr there: https://gist.github.com/zeebo/52f7436e9392cc821b6dfe3af05b5963

Also if you would like, I can probably arrange getting you a shell in to the hardware that is having the issues. Does that sound like a good idea?

@zeebo, yes, that'd probably be helpful, but @ianlancetaylor is on vacation now.
If I ever finish gomote ssh suppport we'd have access to your existing builders. :-/

from dpkg, 2.19-18+deb8u7.

Thanks. Turns out dl-open.c:601 is just the close brace of dl_open_worker. Based on this and the disassembly, this looks like it's just a common epilogue, which is too bad.

https://gist.github.com/zeebo/9bf6059544521f0b311efe4e6953f312

Looks like it's not doing anything funny with LR here. It was pushed on the stack in the prologue and popped from the stack in the epilogue. This suggests either the stack slot got corrupted or the SP itself did.

I took the liberty of running a disas on the contents of lr there: https://gist.github.com/zeebo/52f7436e9392cc821b6dfe3af05b5963

Thanks. This is exactly what I would expect it to be, so the LR is sane on entry.
@zeebo, a shell would be useful. Alternatively, could I get you to break on entry to dl_open_worker again and run this snippet in gdb:
python while True: map(gdb.execute, ["nexti", "bt 1", "info reg", "x/12x $sp"])
It should dump a few thousand lines giving the register state and top-of-stack at every instruction in dl_open_worker so we can trace the exact path through it. You might want to set pagination off and set logging on to capture it.

I ran the python gdb snippet and captured the output here: https://gist.github.com/zeebo/c5b3bd0ff0658132e8794ee39bf3df4a
I've also set you up with a user account on the builder. I used your ssh keys from github, so you should be able to ssh in with

ssh -p 11046<EMAIL_ADDRESS>
There is a checkout of the go repository in ~/go with some modifications to keep the failing binary around, and the binary that is failing is ~/go/misc/cgo/testplugin/host
Let me know if you run in to any issues. I'm also on the gophers slack as zeebo if you want a less asynchronous communication environment.

I ran the python gdb snippet and captured the output here: https://gist.github.com/zeebo/c5b3bd0ff0658132e8794ee39bf3df4a

Perfect. Curiously, it looks like this traced into _dl_init and then failed without getting back out to dl_open_worker. _dl_init makes all sorts of indirect calls to initialization functions provided in tables in the .so, so it wouldn't surprise me if something is slightly off here and tromping on the stack.

I've also set you up with a user account on the builder.

Awesome. Though it's giving me "ssh_exchange_identification: read: Connection reset by peer". If I try connecting directly, I don't get the OpenSSH server identification (tried from two different hosts on very different networks).

I've set up a new relay at relay003b.spacemonkey.com port 12693. Sorry the first one didn't work. :(

Ok, sorry about the relay issues. Maybe try this one relay002b.spacemonkey.com:18268

$ ssh -p 14975<EMAIL_ADDRESS>ssh: connect to host relay005.spacemonkey.com port 14975: Connection refused

:(

Alright, apparently the binary I'm using to forward connections silently decides to stop working after some point in time.
Instead, I have set up a wacky system of ssh reverse tunnels and socat, but maybe it's more reliable. The address is now relay001.spacemonkey.com:7001.

The address is now relay001.spacemonkey.com:7001.

Perfect. That's working for me.
So far all I've managed to figure out is that SP goes bonkers at some point during dl_open_worker, which leads to it restoring a bonkers LR. Strangely, the SP it winds up at is just slightly below the stack of a different M from the one that's running dl_open_worker.
I haven't been able to figure out where it goes bonkers. However, I suspect this is why my Python snippet fails to nexti over the _dl_init.

Found where it goes bonkers as soon as I sent that.
Entering the function epilogue, SP still good:
#0  dl_open_worker (a=<optimized out>) at dl-open.c:584
r0             0xb5d96b98<PHONE_NUMBER>
r1             0x1b7800	1800192
r2             0x1b7800	1800192
r3             0x0	0
r4             0x1da1b0	1941936
r5             0xb6fff050<PHONE_NUMBER>
r6             0x4	4
r7             0x10	16
r8             0xb6fff568<PHONE_NUMBER>
r9             0x194	404
r10            0xb6ffed98<PHONE_NUMBER>
r11            0xb5d95f70<PHONE_NUMBER>
r12            0xb5d40dd0<PHONE_NUMBER>
sp             0xbefff128	0xbefff128
lr             0xb6fe3838<PHONE_NUMBER>
pc             0xb6fe3864	0xb6fe3864 <dl_open_worker+864>
cpsr           0x40000010<PHONE_NUMBER>
0xbefff128:	0xbefff18c	0xb6fe3b18	0x001da1b0	0x00000000
0xbefff138:	0x00000000	0x00000000	0x00000000	0x00000000
0xbefff148:	0x00000000	0xbefff130	0x00000004	0x00000000

Execute ldr	sp, [r11, #-64]	; 0x40 (sp = *(r11 - 64))
#0  dl_open_worker (a=<optimized out>) at dl-open.c:601
r0             0xb5d96b98<PHONE_NUMBER>
r1             0x1b7800	1800192
r2             0x1b7800	1800192
r3             0x0	0
r4             0x1da1b0	1941936
r5             0xb6fff050<PHONE_NUMBER>
r6             0x4	4
r7             0x10	16
r8             0xb6fff568<PHONE_NUMBER>
r9             0x194	404
r10            0xb6ffed98<PHONE_NUMBER>
r11            0xb5d95f70<PHONE_NUMBER>
r12            0xb5d40dd0<PHONE_NUMBER>
sp             0x15d8e8	0x15d8e8 <syscall.statictmp_44>
lr             0xb6fe3838<PHONE_NUMBER>
pc             0xb6fe3868	0xb6fe3868 <dl_open_worker+868>
cpsr           0x40000010<PHONE_NUMBER>

Execute sub	sp, r11, #32 (sp = r11 - 32)
#0  0xb6fe386c in dl_open_worker (a=<optimized out>) at dl-open.c:601
r0             0xb5d96b98<PHONE_NUMBER>
r1             0x1b7800	1800192
r2             0x1b7800	1800192
r3             0x0	0
r4             0x1da1b0	1941936
r5             0xb6fff050<PHONE_NUMBER>
r6             0x4	4
r7             0x10	16
r8             0xb6fff568<PHONE_NUMBER>
r9             0x194	404
r10            0xb6ffed98<PHONE_NUMBER>
r11            0xb5d95f70<PHONE_NUMBER>
r12            0xb5d40dd0<PHONE_NUMBER>
sp             0xb5d95f50	0xb5d95f50
lr             0xb6fe3838<PHONE_NUMBER>
pc             0xb6fe386c	0xb6fe386c <dl_open_worker+872>
cpsr           0x40000010<PHONE_NUMBER>
0xb5d95f50:	0x001c5fd0	0x001b7928	0x001d6408	0x001d63e0
0xb5d95f60:	0xb5da7530	0xb5d6e3b8	0xb5d6e570	0xb5da7519
0xb5d95f70:	0x001ba2e4	0xb6f8fe00	0x001d63d8	0x001d63dc

Next step is to pop the saved registers, so we're supposed to be back at a sane SP now, but aren't.
(I have no idea why it did the intermediate ldr, since the very next instruction clobbered SP.)
It looks to me like r11 is a frame pointer in this function. It uses a variable-length array, so the stack frame is dynamically adjusted at runtime. And the epilogue sequence that restores SP from r11 mirrors the prologue.
r11 got clobbered by the _dl_init call. I'm not sure where yet, but I have a suspicion some .so init function we're generating isn't following the C ABI correctly.

I'm pretty sure I found it, but I managed to completely toast the Go tree you set up for me on the ARM host (oops). @zeebo, would you mind testing https://golang.org/cl/47831?

No problem. I'll give it a shot right now.

It seems fixed. Do we want to revert 168eb9c and let the arm builders chew on it?


Documentation fallback links should include beta releases where no default branch documentation is available

The fallback order should be:

Latest stable release (Highest priority)
Latest pre-release tagged version
Default branch (Lowest priority)

I believe it currently only does:

Latest stable release (Highest priority)
Default branch (Lowest priority)

I’ve had a look around before sending that email and I think we’re already doing that:
https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/blob/02fd647990ed012227a84b1a4a4899bd71ebc36a/Sources/App/Controllers/PackageController%2BPackageResult.swift#L87
It would be odd (or rather a bug I think) if hasDocumentation took pre-releases into account but we then filtered them out in the drop-down.
Worth confirming properly though!

https://swiftpackageindex.com/putridparrot/SwiftUnits

Thanks for checking. I think this does need a tweak. Even though the beta docs do appear, which I didn't doubt they would, the fallback there is back to main.
I think we need to make the fallback go from release, to pre-release, to default branch.

The problem in this case is that there is no main

That's my point, we'd want to fall back to the pre-release before going to the default branch. I think what we currently have would potentially fail in the case where there was a beta tag that had documentation and a main branch that didn't.


Monopoly gives duplicate text when choosing an AI or Player

Describe the bug

To Reproduce
Commands or inputs given before the bug occurred
Error code
Result of [p]traceback (if applicable)
Additional information
Any other relevant information

This can be reproduced by running [p]monopoly twice before doing any other input to the prompt, however since the cog will not allow you to create two games at once anyway, I'm going to say this isn't an issue that needs to be fixed.


Disable tipping of dripleafs for vanished players.

Additions:

Added code to stop drip leafs tipping when vanished
Added the config option to enable/disable this.

Dripleaves were added to the game in 1.17, shouldn't this be using a new feature option called 'oneDotSeventeenOrHigher'? My PR #68 added this feature option but it was not merged in directly, parts of my code were copied over manually it seems, so the option wasn't implemented.

Hey, thanks for your PR! Dripleaves were indeed added later so could you please change that to 1.17?

@ARR4NN Fetch the latest changes on your fork and use the oneDotDeventeenOrHigher requirement, that should do it!

Thank you will do 🫡


add arg to create_image to set output filename

sorry, replied before I'd pushed.  now the push is up.

LGTM


Ability to define index key length while creating schema

Sorry, that I have to open the issue again. Since the issue doesn't get cleared.
Following #9256, what @GrahamCampbell said is not the case. I've quoted here for convience:

I've just tried out v5.1, and got the infamous ERR1071 while doing artisan migrate, right after laravel new. I can't find out a solution to it, yet. Because table charset is utf8mb4, and the email column is (by default) varchar(255), creating a unique on it will definitely cause error. I can't find out a way to specify the key length while calling
$table->string('email')->unique();

from the 2014_10_12_000000_create_users_table.php file.
After a little digging, it seems that we'll have to modify several index related functions in the Illuminate\Database\Schema\Blueprint class, and eventually (maybe not), the Illuminate\Support\Fluent.
Any hint?

@GrahamCampbell answered

Yeh, it's possible. Pass a second param to string with the length.
$table->string('email', 128)->unique();

Continue the talk
Passing in the length to string() changes the length of the column, but not the KEY length. The email column does need to be long enough to hold the data, truncating it to 128 may break the thing.
From what I've dug up:
\Illuminate\Database\Schema\Blueprint::string is defined as:
public function string($column, $length = 255)
{
    return $this->addColumn('string', $column, compact('length'));
}

which just wraps up a call to \Illuminate\Database\Schema\Blueprint::addColumn
protected function addColumn($type, $name, array $parameters = [])
{
    $attributes = array_merge(compact('type', 'name'), $parameters);
    $this->columns[] = $column = new Fluent($attributes);
    return $column;
}

which in turn creates a Fluent instance.
I didn't dig deeper into it, except checking the generated SQL. I can't find any key length related process in the code.
A bit more to clarify, that the key length is specifically MySQL, which makes a partial index using only the start of the column, given length in bytes.

If 128 is too short for you, use something else!

@GrahamCampbell, I think you are misunderstanding @eidng8. As he points out, he does not want to change the length of the column. He wants to change the length of the index prefix, see https://dev.mysql.com/doc/refman/5.0/en/column-indexes.html.

@GrahamCampbell I also think you are misunderstanding @eidng8 and I have a similar requirement for this.
MySQL (and I'm sure other storage engines) support limiting an index not the column by a length. For example for performance purposes you may choose to only index the first 7 characters of varchar column, you may still want a 256 limit but the INDEX needs to be limited.
MySQL SQL code for this could be:
..... .... COLUMN message VARCHAR(300) , ADD UNIQUE INDEX index4 (message(100) ASC);

@GrahamCampbell Please take a look at this again, I think you misunderstood the ticket and closed a perfectly valid feature request. If you re-open the ticket I will implement this myself.

@taylorotwell we can not get an answer from Graham, please look at it.

Indeed, I just received a "Specified key was too long, max key length is 767 bytes" error during a migration. I suspect this is because we recently standardized on utf8mb4 going forward.  Going to talk to DevOps about increasing the max but it would be nice to specify a key length.

+1 on this. It's a problem.

+1

+1

+1, Ran into this problem trying to create an index on a TEXT column.

I pinged @taylorotwell on Twitter to see if this could be re-opened - this would be very helpful for more complex data sets where performance is critical.

I recently faced this issue. A good use case is when you want to store an IPv4/6 address in database  you can use inet_pton to make it binary and inet_ntop to revert it to readable format.
For column type you should use binary and if you want to index this column the length of the index is required.
PS: @taylorotwell it will be nice if you add this common case as native $casts type for IP addresses for Model and change the implementation for $table->ipAddress in migrations to use binary for Laravel 5.5

+1

+1

As a temporary workaround. It is possible to use the following raw syntax for defining indexes etc. $table->index(DB::raw('column(100)'))

That whole migration hassle is so bad, I can't describe how bad :((

This is an issue for one of my projects as well. I have a data that is 255 long and changing the default length to 191 as the laravel manual suggests does not work. Setting the length of the column to 255 is fine but then I cannot index it with without being able to specify and index length of 255.

$table->index([DB::raw('value(100), name')], 'parameters_value_name_index');

@laurencei This can be closed, the PR got rejected.

@myquote-dev Thanks for the suggestion. It works, and it can be used in only the TEXT/BLOB fields.
$table->index([DB::raw('column(100)')])

Should I do anything on the PR then? or is it good enough?

@eidng8 it looks good

Since this is a feature request I'm closing this. Anyone's free to attempt the PR of staudenmeir again.

I wanted to add an index on my text column and ran into this issue.
I was able to work around it by removing the ->index() from the $table->text('some_column') and then added $table->index([DB::raw('some_column(750)')])
I experimented with the number inside the parentheses.. my understanding is that you can only index the first n characters inside a text field. If you go a little higher than 750 you run into this error:
SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 3072 bytes

I wanted to add an index on my text column and ran into this issue.
I was able to work around it by removing the ->index() from the $table->text('some_column') and then added $table->index([DB::raw('some_column(750)')])
I experimented with the number inside the parentheses.. my understanding is that you can only index the first n characters inside a text field. If you go a little higher than 750 you run into this error:
SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 3072 bytes

The most likely reason for this is that 'some_column' is using the utf8mb4 character set which is 4 bytes per character. Therefore the maximum number of characters you can use for your index is 3072 bytes / 4 bytes = 768 characters. utf8mb4 text indexes over 768 will exceed the 3072 byte limit.

@myquote-dev hrm interesting to know.. thanks for explaining
For a few years now I have just been defaulting to using utf8mb4 everywhere, even though in 90% of my projects I only deal with English.. do you think I should re-think my strategy and use regular utf8 instead? Which in this case would allow me to have a larger indexable text/varchar field?

750 should be ample for an index, what you’re after is a low cardinality in the indexes that also doesn’t bloat them (my understanding is the optimiser built in to MySQL will first look to your indexes, but if those indexes are just as big as your data it won’t help quite so much).
If 750 characters isn’t enough length to identify a single record in your table or provide ordering (I expect it is but perhaps not!) then there are other issues with the architecture.
I think you may be stressing over micro optimisations that will probably have little difference on performance - it’s easy enough to test, but I’d expect 750 or 1500 length to provide identical performance. (And for that matter, 250 or less would probably be plenty)

@williamjulianvicary thanks for you input. Well perhaps you can give me your quick advice regarding my specific problem. I am dealing with a database table containing 7 million Chess games. The column I wanted to index in order to speed up queries is a column that contains the Chess line. Here are a few examples of what those values are:
d4d5c4c6Nc3Nf6e3e6Nf3Nbd7Bd3dxc4Bxc4b5Be2Bb7O-OBe7e4
d4e6c4Nf6Nf3Bb4+Nbd2b6a3Bxd2+Bxd2Bb7Bg5d6e3Nbd7Be2O-OO-Oh6Bh4Qe7b4c5Qb3Rab8Rfd1g5Bg3Ne4Nd2Nxg3hxg3d5cxd5Bxd5Bc4Bxc4Qxc4b5Qc3c4Ne4Rfd8a4f5Nc5Nxc5dxc5Rxd1+Rxd1Rd8Rd4e5Rd2a6g4fxg4axb5axb5Rxd8+Qxd8Qxe5g3fxg3
d4Nf6Nf3e6c4b6a3Bb7Nc3d5Bg5Be7Qa4+c6Bxf6Bxf6cxd5exd5g3O-OBg2c5Rd1c4O-Oa6Ne5b5Qc2Qd6e4Bxe5dxe5Qxe5f4Qd6Nxd5Nd7Qf2Rad8Rd2Bxd5Rxd5Qc7Rfd1Nb6R5d4Na4e5Rxd4Qxd4c3bxc3Nxc3Re1Rd8Qe3Nd1Qe2Qc5+Kf1Nc3Qe3Qc4+Kg1a5f5b4e6fxe6fxe6Re8e7Qc8axb4axb4Qd4b3Qb4Kh8Rf1h6Be4Ne2+Kg2g5Qd6

I perform queries such as, show me all lines that begin with d4d5c4c6Nc3Nf6
e.g. SELECT * FROM games WHERE line LIKE "d4d5c4c6Nc3Nf6%"
I know I could throw that up on an Elastic Search instance somewhere etc but right now I'm working with it on my local dev so CPU/RAM is limited and I just wanted to see if adding an index would speed things up. It looks like it did a little bit but it's still quite slow

You could try a shorter index on that field (say 255 characters so you sit within the index length limits of UTF8 or shorter) - that may help ensure your index is able to be used.
If you're requesting X hundred k records with that query though, it's not going to be that quick.

driesvints, Just hiding the feature request doesn't make framework better. PR was rejected, but this feature is still required and would be required. How do people who want improve framework would know, that this is still needed?
You had to link PR to this issue - yes. But not closing it, just because PR exists - it's a worst decision.


Update to latest libcloud release

Codecov Report

Merging #84 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #84   +/-   ##
=======================================
  Coverage   72.29%   72.29%
=======================================
  Files          19       19
  Lines        1256     1256
=======================================
  Hits          908      908
  Misses        348      348

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6510dcc...f04a7cd. Read the comment docs.


Dataset to TFRecord

The dataset API is excellent, I am even using it to perform data augmentation before training. However, I notice that when converting my dataset to tfrecord I follow a pattern that might easily be automated as a to_records method, in fact I think this is somewhat similar to using the cache method. Anyway, it would make it very straight forward for dataset users to create tfrecords.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

This is a fine suggestion, and something we have been discussing internally. I'm assigning this issue to @jsimsa, who has expressed interest in developing it.
@cgarciae There are a lot of possible designs for a feature like this. Have you had any thoughts on what your ideal API would look like for this task?

@mrry Very glad you are thinking about this!
The signature could be to_records(self, features, ...) where:

It follows the same API of tf.parse_example so you can reuse the same concepts/structures.
Only work on dict-based datasets so you can match the structure in the features dict to the structure of the dataset dict when creating the tf.Example. People who use the tuple-based datasets can just map to a dict beforehand.
Create a tf.ImageFeature class that indicates you want to convert the ndarray to an encoded image in bytes.

Btw: I find that tuple-based datasets are harder to maintain when you have many features/columns, the documentation should switch to dict-based approach, or you should even provide a Row object like pyspark so we can access the fields using ..

Nagging Assignee @jsimsa: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

I guess this functionality would simplify a lot and encourage the usage TFRecord files. Furthermore, the general documentation on creating/writing to TFRecord files is still sparse.
Was there some progress on this issue?

@cgarciae Could you post an example of how you convert a Dataset to TFRecords?
I tried this hacky approach:
def to_tfrecords(ds, fn):
    ds = ds.cache(fn)
    iterator = ds.make_one_shot_iterator()
    features = iterator.get_next()

    with tf.Session() as sess:
        while True:
            try:
                sess.run(features)
            except tf.errors.OutOfRangeError:
                break

where ds is a Dataset and fn is a file path. This creates fn + ".data-00000-of-00001" and fn + ".index" files successfully, but when I try reading it with tf.python_io.tf_record_iterator I get a DataLossError: corrupted record at 0 (TF 1.7). Perhaps it's not actually a TFRecords file or is up somehow, but I haven't found much documentation about cached datasets.

Nagging Assignee @jsimsa: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@cgarciae thanks for that, I'm using it!
I suppose one could pass parse_fn as an argument:
def to_tfrecord(ds, parse_fn, record_path):
    with tf.Session() as sess, tf.python_io.TFRecordWriter(record_path) as writer:
        generator = dataset_generator(ds, sess)

        for row in generator:
            example = parse_fn(row)
            writer.write(example.SerializeToString())

Nagging Assignee @jsimsa: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 44 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 59 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 74 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 89 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 104 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 119 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 134 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 149 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 164 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @jsimsa: It has been 179 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

On a related note that the updated tf.Example tutorial illustrates how to use tf.data for both reading and writing TFRecord files, which is recommended over using tf.python_io.

Hi,
Why this issue closed? I did not see the problem solved, if not could anyone point me to the API?
@jppgks
Your solution is really good, write once use anywhere. It looks like your solution is only worked for eage execution.
I try hard to bypass creating tf records for cloud TPU, but failed. So looks I must do. I hope I can have a solution work well for different projects when building different models, generally speaking, when I build a model, I will create list of input features(named tuple). It work well for the TF API tf.parse_single_example, I can create the record_spec like below:
for fea in self._input_features:
     record_spec[fea.name] = tf.FixedLenFeature(fea.shape, fea.dtype).
tf.parse_single_example(tfrecord, record_spec)

So if the dataset have an API like tf.data.Dataset.to_tfrecords(record_spec, fpath) would be great.
Follow jppgks, I can create a function like below, but the problem is it create lots of tenors in the graph.
def ds2tfrecord(ds, filepath):
    with tf.python_io.TFRecordWriter(filepath) as writer:
        itr = ds.make_one_shot_iterator()
        with tf.Session() as sess:
            try:
                while True:
                    feat_dict = itr.get_next()
                    features = {}
                    for name, fea in feat_dict.items():
                        serialized_tensor = tf.serialize_tensor(fea)
                        bytes_string = sess.run(serialized_tensor)
                        bytes_list = tf.train.BytesList(value=[bytes_string])
                        features[name] = tf.train.Feature(bytes_list=bytes_list)
                    # Create a Features message using tf.train.Example.
                    example_proto = tf.train.Example(features=tf.train.Features(feature=features))
                    example_string = example_proto.SerializeToString()
                    # Write to TFRecord
                    writer.write(example_string)
            except tf.errors.OutOfRangeError:
                pass

You can track https://github.com/tensorflow/tensorflow/issues/38483


change path to sphinx conf.py

Fixes a build issue with ReadTheDocs where the Sphinx conf.py configuration file couldn't be found.
https://readthedocs.org/projects/rashdf/builds/24604364/

Skipping review; iterating quickly to get ReadTheDocs builds working. No code changes.


Binding to UserControl.IsVisible and setting DataContext breaks binding

When binding IsVisible (didn't check other properties) of an UserControl (didn't check other controls) and setting the DataContext, the IsVisible binding won't work. Not setting the DataContext works fine.
Example
View:
<StackPanel>
  <ToggleButton IsChecked="{Binding ShowUC}" />
  <UserControl DataContext="{Binding MyDataContext}" IsVisible="{Binding ShowUC}">
    Hello World!
  </UserControl>
</StackPanel>

View Model:
private bool _showUC;

public bool ShowUC
{
    get => _showUC;
    set => this.RaiseAndSetIfChanged(ref _showUC, value);
}

public object MyDataContext { get; } = new object();

Is this a bug or am I missing something?
Any help would be appreciated.

The DataContext is used for all the other bindings on the control so it does not work because ShowUC is not a property of MyDataContext. $parent.DataContext.ShowUC should work

That works, but I feel that the behavior is not really intuitive... I would expect all bindings, except the ones in templates, to use the root element DataContext as the default one, but WPF has the same behavior, so I'm probably missing the point.

I was having this issue when I found the solution stated above which worked fine.
However, here and in the Avalonia-UI documentation, I find a lack of context in the examples/solutions.
I see lines of code that doesn't fully explain where they belong and I end up having to try different slightly different variations because every dot and comma does matter.
So I will add here the full working example for this issue as a reference for others.
Fixed Example:
MyView.axaml:
<Window ...
  xmlns:views="clr-namespace:MyProject.Views;assembly=MyProject" />

...
<StackPanel>
  <ToggleButton IsChecked="{Binding ShowUC}" />
  <views:UserControl DataContext="{Binding MyDataContext}" IsVisible="{Binding $parent.DataContext.ShowUC}">
    Hello World!
  </views:UserControl>
</StackPanel>

MyViewModel.cs:
private bool _showUC;

public bool ShowUC
{
    get => _showUC;
    set => this.RaiseAndSetIfChanged(ref _showUC, value);
}

public object MyDataContext { get; } = new object();


feat: development build switch

Allows the user to be able to switch between development and stable builds easily.

New interfaces/APIs aren't exposed to plugin developers, but that will probably change after #29 is completed.

Need to test on iOS & fix reload popup not appearing after toggling once

All things seem to work now, just need to test on iOS.


Update core.py to match up-stream

jcarbaugh has added a few new commands to his list, and it would be great to have them available!
https://github.com/jcarbaugh/python-roku/blob/5b12c15777baf77424b525bdd469509906c13ee7/roku/core.py

I have no problem merging this however if this is for home assistant can you verify if using jcarbaugh pypy image in homeassistant works without much changes. I want to eventually merge this back into his original work so if it's at a point that that will work we may want go that route.
If not I will merge and push a new pypi package

This is for home assistant, so I'm willing to try that. Do you have any pointers on how to pull the master branch of his package in? It looks like he hasn't pushed a new version to pypi since 2014. https://pypi.python.org/pypi/roku

I have requested a new release at https://github.com/jcarbaugh/python-roku/issues/28

That's probably the best solution to this, hopefully it works!


Link to  Law Enforcement Requests Report is not working

In Chapter "What third parties have access to personal data?" the link to  Law Enforcement Requests Report is not working
https://www.microsoft.com/about/corporate-responsibility/lerr/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e34f35bb-7d9c-0c7f-a1a1-6fd9fbe93dfa
Version Independent ID: d123210a-ed20-4b97-3706-33d7887b4c27
Content: Microsoft Teams Privacy - Microsoft Teams
Content Source: Teams/teams-privacy.md
Service: msteams
GitHub Login: @MicrosoftHeidi
Microsoft Alias: heidip

I've updated the URL, it should be merged once it's approved. https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/pull/5192

@Florian-Mayer Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.


Global & Workspace View: View Linearity Summary Data

User Story
As an ECMPS user, I want a new QA - Linearity Summary Data section so that I can reference it when making updates
Acceptance Criteria
Given a user selects Linearity Summary from the Test Type dropdown (Login for the Workspace)
When the Linearity Summary Data is displayed
Then the data is displayed in a single table (see list below for columns and order)
Gas Level Code
Mean Measured Value
Mean Reference Value
Percent Error
APS Indicator
Given a user is viewing Linearity Summary Data
When the data is displayed in a table
Then the default sort order is alphanumeric descending by the contents in the 1st column
And the user will have the option to sort the columns in ascending or descending alphanumeric order
Note: For the Global-view, it shall be a read-only version without the ability to edit or create
For the Workspace, the user shall have the ability to View and Edit Linearity Summary Data

[x] The fields and their order specified

Pagination?

[ ] Yes
[x] No

Filter by keyword?

[ ] Yes
[x] No

Columns are sortable?

[x] Yes
[ ] No

Nested rows, expandable?

[x] Yes
[ ] No

Reporting Instructions (Page 32): https://ecmps.camdsupport.com/documents/ECMPS QA Reporting Instructions 2021 Q4.pdf
Invision Board URL: https://projects.invisionapp.com/freehand/document/TdahPfbEb
Reference Screenshots available in the comments section
Testing
Test Case: TC797
Tester: @JanellC

ECMPS-QA Linearity - Desktop large - 1024 V3.0
Xd prototype: https://xd.adobe.com/view/b7d2b53e-ef66-4e08-92f4-79ce5de9d33b-e281/screen/e8711449-8bbb-4419-96ed-9b589459797c

TC797 passed
Global

Workspace