Why dev-1.18 keeps generating merge conflicts

What's the problem?
I notice that the dev-1.18 branch repeatedly generates merge conflicts for unrelated files, often for the same file (/content/en/docs/reference/kubectl/overview.md), every time a PR merges into the branch.
Example
Here's the timeline view:

Here's the commit view:

The commit for #19008 generates merge conflicts for unrelated files:

To break this down:

I merged master into dev-1.18, which generated merge conflicts. https://github.com/kubernetes/website/pull/19116/commits/78193337e445629b9f72bc2920a04ead003bb6ff

I added a commit resolving the merge conflicts generated by https://github.com/kubernetes/website/pull/19116/commits/78193337e445629b9f72bc2920a04ead003bb6ff, and restored the branch to health. https://github.com/kubernetes/website/pull/19116/commits/4582f017e27aa19fe98068765eac4cd2f8f9078e

#19008 merges into dev-1.18 and generates merge conflicts for unrelated files.

This is one example of a larger pattern in dev-1.18 beginning with the erroneous merge in #19055 and subsequent attempts to repair the branch.
What will happen if we don't fix it?
I think that new merge conflicts will generate every time a feature PR merges into dev-1.18 unless every open PR based on dev-1.18 rebases.
Currently that's 32+ PRs.
How do we fix it?
Every open PR based on dev-1.18 needs to rebase.
Yes, that's a lot. Feature developers will probably grumble.
It's better than having to resolve (the same) merge conflicts every time a PR merges.
Does this seem accurate? For review:
@VineethReddy02 @jimangel @sftim @kbhawkey

If rebasing fixes (permanently) the problem(s), then I think that is a favorable outcome.
I don't think there will be grumbling (I would gather that most developers rebase regularly).

Every open PR based on dev-1.18 needs to rebase.

Yes, that's quite a lot. It does seem like the best approach.

@sftim @kbhawkey Thanks for the feedback.
@VineethReddy02 To make the rebase requests easier, here's some sample text to comment on open PRs:
👋 Please rebase this PR on `dev-1.18` in order to avoid merge conflicts.

Feel free to `/hold cancel` after you've rebased.

/hold

Feel free to adjust to your liking. 👍 Given the number of comments, if you're comfortable working with the GitHub API, it may be worthwhile to add comments with a script. If not, manually adding them is OK too.

Bless this Prow. ⛵ ☸ 🙏 😂

/hold cancel

@kubernetes/sig-release @alejandrox1  👋 Heads up that this is a thing.

/sig release

Hi @zacharysarah @sftim but I already see /hold label for some other reasons by reviewers and owners. How will the behavior be if I add something like this
👋 Please rebase this PR on `dev-1.18` in order to avoid merge conflicts.

Feel free to `/hold cancel` after you've rebased.

/hold

/hold cancel

This is an issue; the /hold command and associated label are (I think) mainly for PRs.
If I'm right @zacharysarah was adding /hold here merely to show the sample text.
Prow doesn't stack /hold labelling, so if the PR is already held for one reason I would adjust the message to say something like:

Feel free to /hold cancel once the upstream code is merged and you've rebased your changes.
If you have rebased but the upstream code is not yet merged, please leave the /hold in place and
add a note to let SIG Docs know you have done that.

Thanks, @sftim @zacharysarah I reached out to all 31 open PR's against dev-1.18 and I see 4 PR's already with /hold label. So I don't think it should be a problem for us. However, I can try to have a close eye on them.

xref: https://github.com/kubernetes/test-infra/pull/16892
(which I think will solve a chunk of the problem here)

+1 for avoiding this issue with proper dev-branch management. Closing for now.
/close


Some US holidays improvements

St. Patrick's day, Earth day, Presidents day, Cyber Monday, Veterans day. To @wlifuture

Resolving by PR #1617 https://github.com/microsoft/Recognizers-Text/pull/1617

Patch previous PR #1613 that initially improve two holidays

And latest "easter" + related special days which are based on lunar calendar

@wlifuture are more coming in the near future? Or can we close this issue for now?

@wlifuture are more coming in the near future? Or can we close this issue for now?

Closing the issue for now.


Unused parameter "custom_payload" in TelegramAction

TelegramAction has a parameter "custom_payload" that is not used anywhere. It should either be used or dropped.
https://github.com/d-Rickyy-b/pastepwn/blob/d37af685bc452c8c2f88b7443e001782a9b1fc7d/pastepwn/actions/telegramaction.py#L13

Fixed in 7b13d757242af33ddc9ce1c2c3e5e3d7618cd4b5


No ptc2depth !

Hello, thanks for your work. But when I run 'bash prepare_4beam_data_for_prediction.sh', the gen2channel.py can not import ptc2depth form kitti_utils.

Hi Weixu:
I just updated the kitti_utils.py, can you try again?
Sorry for the inconvenience

Thank you for your replay. However, it still lacks added_minus1.txt in splits/eigen, does it need to execute other files to get it ?

I think I messed it with another work, the 'added_minus1.txt' is not needed, thank you for pointing it out
I have updated the gen2channel.py and kitti_utils.py. Please let me know if there are any other problems

Hello， when I run evaluate_depth.py, I found there is no ggdc_old, is there something wrong?

Hi Weixu:
Seems I missed the gdc_old.py, I have uploaded it


Add fallback input option to ensure that fallback is not used

Add an option to prevent accidentally installing tools that use fallback for installation.
- uses: taiki-e/install-action@v2
  with:
    tool: <tool>
    # Possible values:
    # - none: disable all fallback
    # - cargo-binstall (default): cargo-binstall (includes quickinstall)
    fallback: none

We could add an option to use cargo-binstall without quickinstall in the future if someone actually wants to use that.
Related discussion: https://github.com/taiki-e/install-action/pull/514
cc @jayvdb

Considering there is a proposal to add a non-cargo-binstall fallback (https://github.com/taiki-e/install-action/issues/526) too, I think this option is good as is. We can discuss how to pass additional flags to cargo-binstall in another issue.


Error with RPC Calls and select queries with postgrest v0.16.9

Bug report
Describe the bug
This started arising with postgrest v0.16.9 only, it was working perfectly with version v0.16.8
To Reproduce
Steps to reproduce the behavior, please provide code snippets or a repository:

Make a database function with parameter
run it with supabase.rpc in v2.6
Nothing will be returned from the db, probably malformed parameter
See error

Expected behavior
It should work and proper response from db should be returned
Screenshots
System information

OS: docker
Version of supabase-py: v2.5+

Please show code example when opening issues to do with code bugs. I've tested this and cannot replicate the issue you are having. I've attached a video showing this working. I'm also using the latest 0.16.9 version of the postgrest library.
https://github.com/user-attachments/assets/fd149b53-dc4e-4bdc-a56d-241d9a18ac88

Thank you for checking it out.
I will check in with my container and share code example soon.

Closing this out as there has been no reply from OP and I've shown this working in the video above.


Request for documentation

I'd like to learn more why I should start with "suiet wallet-kit" and not other options.
Mainly I want to understand what the relation/comparison is between "suiet wallet-kit" and
sui wallet-standard
sui wallet-kit
ethos connect

Hi, @georgescharlesbrain . Basically, these are three kits that all implement the Sui wallet standard, so you are good to try with either of them. The key difference among them I would say is the style of out-of-the-box UI components and the API design. So you can choose them based on your programming preferences and product design needs.

Thanks will look into it further then.


Nrf5340 netcore

Added support for the nrf5340-netcore. This support consists mostly of cfg annotations and some very small adjustments.

Hey, would love to see this merged. @Sandraak is an intern with us at Tweede golf and she's been working on this under my guidance. She has also tested some of the peripherals to check if everything is working alright and it did.
I think it's good enough to be merged as it is right now.

Taking a look :eyes:

Thanks for the feedback @jonas-schievink !

It looks like your commit email isn't set to one that GitHub recognizes. If you want your contribution to be attributed correctly, consider adding your commit email to your GitHub profile.

bors r+


Fix typo "that are will" -> "that will"

@bsquizz @psav not sure if the changed files are related to this fix :thinking:

You're right, the doc changes looked a little larger than you probably intended. Something is probably different on your environment that was causing the docs to be generated with slight formatting differences. I re-generated the API docs.


Remove unused header parameters from render tables

The config_set and num_col header parametrers are not currently used for anything and, as far as I can tell, have no planned use either.
The latter can only be found in the Vue app example test data. These references should all be removed.

Done, to be merged.


keybindings for go testing?

How to find out what keybindings are available for Go testing when editing go code.

I tried to do a fresh install to fix this and I get the following error message after running nvim without any parameters.
Error detected while processing /home/aphsa/.config/nvim/init.lua:
Plugin telescope-file-browser.nvim is not installed
Press ENTER or type command to continue

Hello,
There are no default keybindings (yet) but you can use ex commands or define your own.

The plugin is called gopher.nvim and is disabled by default
return {
  plugins = {
    gopher = {
      enable = true,
    },
}

In general, you can enter :map to get a list of all keybindings (or its variants nmap, imap, vmap)

That worked.


select linkage

Laravel Version: 5.5.*
PHP Version: 7
Laravel-admin: 1.5.*

Can I use select linkage in "Filter" ?

Do you mean cascading selects?

Example - I have a lot of "villages" list in db. I want to filter villages by "state" and "townships of selected state".

ref http://laravel-admin.org/docs/#/en/model-form-fields?id=select-linkage

@z-song
I use Select Linkage in Filter. As below example:
http://laravel-admin.org/docs/#/en/model-form-fields?id=select-linkage
When I selected province, the default option of city is the first option. How can I set default option is empty?


feat(customLayout): make defineLayout available via customLayout prop

Hi,
Thanks for creating this library. It has been super useful. Added a change to expose defineLayout from PptxGenJS by adding a optional prop called customLayout
This PR addresses this issue, https://github.com/wyozi/react-pptx/issues/663

@wyozi Made the changes as suggested! Thanks for taking time to review. :)

@cpappala-godaddy @ChaitanyaKrishnaPappala can't see the new changes at least on this PR 😅

@wyozi Sorry it was on my local branch, pushed it now.

Thanks 👍


fix the old test case record.getComment() will never be null

The old test case record.getComment() will never be null and if record.getComment() be null the test code misplace the null test.
Add a new test file that record.getComment() will be null and test record.getComment() no null before using.

Coverage remained the same at 95.323% when pulling 9bc2d808c998db13d4774b48eccaee87526d2d88 on dota17:FixCSV246 into ea7ffd7b8ee3da84e6cc4d97c110a9bea05633b6 on apache:master.


[2.0] Class binding removes externally added class

Vue.js version
v2.0.0-rc.5
Reproduction Link
https://jsfiddle.net/h7mz81oo/5/
Steps to reproduce

Add a :class binding to an element, and also add a class to that element using DOM methods (this.$el.querySelector('span').className += " external-class";).

Update the :class binding

What is Expected?
The class added using DOM methods should sill be present on the element along with the class added via vue binding
What is actually happening?
The class added using DOM methods is removed and only the class added via vue binding remains

This is expected behavior in 2.0. If you use Vue to manage the class then all class manipulation needs to happen theough Vue.

@yyx990803 , i think, it will be better if we only handle the bounded className under vuejs management. In many cases, one tag has some other pre-populated class which is not under vuejs control. The behaviour works perfect under vuejs1.0


remove hard-coded local repository

@yungez @xscript @yaweiw

@ZhijunZhao,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


chore(deps): update renovate/renovate docker tag to v35.6.0

This PR contains the following updates:

Package
Update
Change

renovate/renovate
minor
35.4.1 -> 35.6.0

Release Notes

renovatebot/renovate
v35.6.0
Compare Source
Features

auto-replace: ability to specify global or first match from replacement rules (#20604) (8137533)

v35.5.0
Compare Source
Features

add datasource, versioning and manager for Azure Bicep resources (#20755) (3c5224e)

Bug Fixes

presets: add k8s registry move to all replacements (#20928) (b748131)

v35.4.2
Compare Source
Bug Fixes

log duration after rawExec error (#20924) (d458a93)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 4.4.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Does it have support for mobile/tablet?

Hello. Your question is not very clear. If you want to know if Robot Framework is able to test a mobile/tablet application, yes it is possible with libraries like https://github.com/serhatbolsu/robotframework-appiumlibrary. Note that this "issue@github" section is for bug or enhancement requests, if you have question, please use mailing list or slack.


Mobile_fu shows Mobile Site to Search Bots (googlebot) instead of Desktop Site

Mobile_fu shows Mobile Site to Search Bots (googlebot) instead of Desktop Site. This is causing my site not to get indexed properly by google. For example when you do a site:MYSITE.COM in google look at the first link, homepage.. and select "cached" from google's search result page you will see that it is caching my mobile version of my home page instead of the desktop version.

This seems to be a Rails 3 peculiarity, not explicitly a mobile_fu thing. If request.format is nil (for example, when type / is requested, which is what Googlebot asks for), Rails prefers templates that match .mobile.erb over .rhtml. Changing our (very old) file extensions from .rhtml to .html.erb fixed this problem for us.


Upload Issue: Cannot find nested property 'oids' on the resource type 'search.documentFields'.

Please provide us with the following information:

This issue is for a: (mark with an x)
- [X] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Upload file after bootup with necessary environmental varibale changes such as AZURE_AUTH and the others detailed in the readme.

Any log messages given by the failure

2024-12-10 18:29:01,105 - INFO - Ingesting 'Assessment Guide.pdf'
2024-12-10 18:29:01,107 - INFO - Extracting text from 'Level 1 CMMC SELF-Assessment Guide.pdf' using Azure Document Intelligence
2024-12-10 18:29:12,902 - INFO - Splitting 'Level 1 CMMC SELF-Assessment Guide.pdf' into sections
2024-12-10 18:29:12,947 - INFO - Section ends with unclosed figure, starting next section with the figure at page 2 offset 1684 figure start 840
2024-12-10 18:29:12,948 - INFO - Section ends with unclosed figure, starting next section with the figure at page 2 offset 3585 figure start 938
2024-12-10 18:29:12,968 - INFO - Section ends with unclosed figure, starting next section with the figure at page 52 offset 93012 figure start 860
2024-12-10 18:29:13,448 - INFO - Computed embeddings in batch. Batch size: 16, Token count: 4173
2024-12-10 18:29:13,686 - INFO - Computed embeddings in batch. Batch size: 16, Token count: 3703
2024-12-10 18:29:13,907 - INFO - Computed embeddings in batch. Batch size: 16, Token count: 3506
2024-12-10 18:29:14,126 - INFO - Computed embeddings in batch. Batch size: 16, Token count: 3708
2024-12-10 18:29:14,341 - INFO - Computed embeddings in batch. Batch size: 16, Token count: 3683
2024-12-10 18:29:14,545 - INFO - Computed embeddings in batch. Batch size: 16, Token count: 3455
2024-12-10 18:29:14,648 - INFO - Computed embeddings in batch. Batch size: 3, Token count: 1061
2024-12-10 18:29:15,132 - ERROR - Exception on request POST /upload
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1403, in handle_request
return await self.full_dispatch_request(request_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1441, in full_dispatch_request
result = await self.handle_user_exception(error)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1029, in handle_user_exception
raise error
File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1439, in full_dispatch_request
result = await self.dispatch_request(request_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1535, in dispatch_request
return await self.ensure_async(handler)(**request_.view_args)  # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/decorators.py", line 56, in auth_handler
return await route_fn(auth_claims, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/app.py", line 374, in upload
await ingester.add_file(File(content=file_io, acls={"oids": [user_oid]}, url=file_client.url))
2024-12-10 18:29:15,140 - <IP_ADDRESS>:0 - "POST /upload HTTP/1.1" 500
File "/app/prepdocslib/filestrategy.py", line 144, in add_file
await self.search_manager.update_content(sections, url=file.url)
File "/app/prepdocslib/searchmanager.py", line 297, in update_content
await search_client.upload_documents(documents)
File "/usr/local/lib/python3.11/site-packages/azure/search/documents/aio/_search_client_async.py", line 578, in upload_documents
results = await self.index_documents(batch, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/azure/core/tracing/decorator_async.py", line 105, in wrapper_use_tracer
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/azure/search/documents/aio/_search_client_async.py", line 677, in index_documents
return await self._index_documents_actions(actions=batch.actions, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/azure/search/documents/aio/_search_client_async.py", line 685, in _index_documents_actions
batch_response = await self._client.documents.index(batch=batch, error_map=error_map, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/azure/core/tracing/decorator_async.py", line 105, in wrapper_use_tracer
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/azure/search/documents/_generated/aio/operations/_documents_operations.py", line 857, in index
raise HttpResponseError(response=response, model=error)
azure.core.exceptions.HttpResponseError: () The request is invalid. Details: Cannot find nested property 'oids' on the resource type 'search.documentFields'.
Code:
Message: The request is invalid. Details: Cannot find nested property 'oids' on the resource type 'search.documentFields'.
Expected/desired behavior

Just wanted to upload a document for a summary.

OS and Version?

My laptop is Windows 11, but if that isn't the OS you're looking for let me know.

azd version?

run azd version and copy paste here.
azd version 1.11.0 (commit 5b92e0687e1fa96dfc8292f4b900c0c58610b6a5)

Versions

Mention any other details that might be useful
Was trying to upload document and got the error  in the popup. After a couple minutes the document popped up as if it had actually been successfully uploaded, but the model couldn't reference the document for inference.

Thanks! We'll be in touch soon.

Did you follow these steps from the docs after enabling the upload feature?
If you are enabling this feature on an existing index, you should also update your index to have the new storageUrl field:
python ./scripts/manageacl.py  -v --acl-action enable_acls

And then update existing search documents with the storage URL of the main Blob container:
python ./scripts/manageacl.py  -v --acl-action update_storage_urls --url <https://YOUR-MAIN-STORAGE-ACCOUNT.blob.core.windows.net/content/>

Great, no worries!


Is the gem working ?

Getting this error for all the requests with valid credentials
 {"errors":[{"id":"f2b6a3ee-9d8b-45f1-87a8-e035418cd5c1","status":401,"code":"not_authenticated","title":"Authentication credentials were not provided.","detail":"Missing or invalid authorization scheme. Please use Klaviyo-API-Key.","source":{"pointer":"/data/"}}]}

You might be missing the key prefix Klaviyo-API-Key, which makes your api key look something like this:
# Setup authorization
KlaviyoAPI.configure do |config|
config.api_key['Klaviyo-API-Key'] = 'Klaviyo-API-Key PKHERE'

If this isn't the issue, can you post a small snippet of some of your code (With api key removed).

Yes, the gem works.
The redundant Klaviyo prefix also through me for a bit.
This does work though:
KlaviyoAPI.configure do |config|
  config.api_key['Klaviyo-API-Key'] = "Klaviyo-API-Key #{Rails.application.secrets.klaviyo_api_key}"
end


WIP: Test on multiple SQLite version

📚 Documentation preview 📚: https://datasette--2352.org.readthedocs.build/en/2352/

Looks like that test failure is caused by:

#2353

    async def test_derive_named_parameters(sql, expected):
        ds = Datasette([], memory=True)
        db = ds.get_database("_memory")
        params = await utils.derive_named_parameters(db, sql)
>       assert params == expected
E       AssertionError: assert ['00', 'cat'] == ['cat']
E
E         At index 0 diff: '00' != 'cat'
E         Left contains one more item: 'cat'
E
E         Full diff:
E           [
E         +     '00',
E               'cat',
E           ]

That test is still failing because the whole point of that test is to confirm that the clever trick I came up with (and that gets skipped on SQLite 3.46.0) correctly avoids the "0:00" trick string.

Let's drop macOS for the moment, and just go with Linux.

Got this intermittent failure:
    def test_max_csv_mb(app_client_csv_max_mb_one):
        # This query deliberately generates a really long string
        # should be 100*100*100*2 = roughly 2MB
        response = app_client_csv_max_mb_one.get(
            "/fixtures.csv?"
            + urllib.parse.urlencode(
                {
                    "sql": """
                select group_concat('ab', '')
                from json_each(json_array({lots})),
                    json_each(json_array({lots})),
                    json_each(json_array({lots}))
                """.format(
                        lots=", ".join(str(i) for i in range(100))
                    ),
                    "_stream": 1,
                    "_size": "max",
                }
            ),
        )
        # It's a 200 because we started streaming before we knew the error
>       assert response.status == 200
E       assert 400 == 200
E        +  where 400 = <datasette.utils.testing.TestResponse object at 0x7fd35cdc04d0>.status

Let's comment out that test so it doesn't block us on merging this PR, then fix that test in this issue:

#2355

Flaky test failed again in a different way:
    def test_max_csv_mb(app_client_csv_max_mb_one):
        # This query deliberately generates a really long string
        # should be 100*100*100*2 = roughly 2MB
        response = app_client_csv_max_mb_one.get(
            "/fixtures.csv?"
            + urllib.parse.urlencode(
                {
                    "sql": """
                select group_concat('ab', '')
                from json_each(json_array({lots})),
                    json_each(json_array({lots})),
                    json_each(json_array({lots}))
                """.format(
                        lots=", ".join(str(i) for i in range(100))
                    ),
                    "_stream": 1,
                    "_size": "max",
                }
            ),
        )
        # It's a 200 because we started streaming before we knew the error
>       assert response.status == 200
E       assert 500 == 200
E        +  where 500 = <datasette.utils.testing.TestResponse object at 0x7fe7fb6d0790>.status

/home/runner/work/datasette/datasette/tests/test_csv.py:213: AssertionError
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/home/runner/work/datasette/datasette/datasette/database.py", line 306, in sql_operation_in_thread
    cursor.execute(sql, params if params is not None else {})
sqlite3.OperationalError: interrupted

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/datasette/datasette/datasette/app.py", line 1732, in route_path
    response = await view(request, send)
  File "/home/runner/work/datasette/datasette/datasette/app.py", line 1899, in async_view_for_class
    return await async_call_with_supported_arguments(
  File "/home/runner/work/datasette/datasette/datasette/utils/__init__.py", line 1022, in async_call_with_supported_arguments
    return await fn(*call_with)
  File "/home/runner/work/datasette/datasette/datasette/views/base.py", line 89, in __call__
    return await handler(request, datasette)
  File "/home/runner/work/datasette/datasette/datasette/views/database.py", line 61, in get
    return await QueryView()(request, datasette)
  File "/home/runner/work/datasette/datasette/datasette/views/base.py", line 89, in __call__
    return await handler(request, datasette)
  File "/home/runner/work/datasette/datasette/datasette/views/database.py", line 564, in get
    return await stream_csv(datasette, fetch_data_for_csv, request, db.name)
  File "/home/runner/work/datasette/datasette/datasette/views/base.py", line 441, in stream_csv
    response_or_template_contexts = await fetch_data(request)
  File "/home/runner/work/datasette/datasette/datasette/views/database.py", line 560, in fetch_data_for_csv
    results = await db.execute(sql, params, truncate=True)
  File "/home/runner/work/datasette/datasette/datasette/database.py", line 336, in execute
    results = await self.execute_fn(sql_operation_in_thread)
  File "/home/runner/work/datasette/datasette/datasette/database.py", line 282, in execute_fn
    return await asyncio.get_event_loop().run_in_executor(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/runner/work/datasette/datasette/datasette/database.py", line 280, in in_thread
    return fn(conn)
  File "/home/runner/work/datasette/datasette/datasette/database.py", line 319, in sql_operation_in_thread
    raise QueryInterrupted(e, sql, params)
datasette.database.QueryInterrupted: (OperationalError('interrupted'), "\n            select group_concat('ab', '')\n            from json_each(json_array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99)),\n                json_each(json_array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99)),\n                json_each(json_array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99))\n            ", {'_stream': '1', '_size': 'max'})

Split out another flaky test issue:

https://github.com/simonw/datasette/issues/2356


Latest version of ion-sdk-js package does not emit events for stream-add rtc messages

Your environment.

Version: >= 1.5.3
Browser: include version
Other Information - stacktraces, related issues, suggestions how to fix, links for us to have context

What did you do?
I setup an ion cluster, ran a javascript app using the ion-sdk-js package, and had two peers join the call to see if they can subscribe to each others' stream.
What did you expect?
I expect there to be a way to hook into stream-add rtc messages. They include stream IDs which I believe are what should be used when subscribing to a peers' stream.
At the moment, the library only provides access to new peers through the ontrack method which is emitting different stream ids than the stream-add rtc messages.

Why are the stream ids different when emitted from ontrack and stream-add rtc messages?
Which one should the client actually subscribe to?
Can we please add a callback for a the stream-add rtc message? (I can submit a PR for this)

What happened?
The stream-add rtc event is ignored and the only accessible callback is the ontrack callback which may be emitting stream ids which are not valid (within the context of the ion sfu).

cc: @furqanrydhan

I think this should support in pion/ion biz module


Add AR hit test support to status.

CC: @petele
[WF_IGNORE:MAX_LEN,SCRIPT]

Just staging this for Thursday.

:+1:

:+1:

:+1:

:+1:

:+1:

:+1:

@petele This one is ready for you.

"Do you also want to use this opportunity to move WebVR/AR to a top level node?"
Not yet.
Also, PTAL.

:+1:

:tada: This has been pushed live to https://developers.google.com/web/


new voucher QR type

A voucher QR contains an account's secret key.
Useful for cashback at stores (pay in national currency, get back a bit of CC)

create fresh QR with freshly generated account
a store would scan the empty voucher and can transfer funds to the voucher. A hand-written note on the voucher would indicate the value of the voucher
the recipient of the voucher would scan the QR code and the app

verifies the community (and allows to set the resepective community if not already selected)
asks the user to choose the benficiary account
immediately reaps the account entirely
forgets the voucher data

QR format
For compact QR codes, we should rather use hex-encoded secret seeds. No need to export these accounts in the app anyway as they never become part of the profile accounts
encointer-voucher
V1.0
0xaa8695db51801877fc4eab022df3d914513fca24c60b9ed97ec98535fe487f96
sqm1v79dF6b
Name of issuer

Looks good from the design perspective, we have two technical difficulties here, we need to investigate:

I don't know yet how to get the tx fee estimation in CC, so I don't know how to calculate how much to send, to reap the account, but still send enought CC. We could however easily derive a something on our own based on our estimates, accepting potentially a bit of funds burned
On the pallet side, we don't reap accounts yet, i.e., the balance is allowed to be 0. The ED is currently only considered, when creating new accounts.

I believe we should add a new dispatchable: transfer_all
Solves the problem cleanly

We cannot assume that the recipient of the voucher already has the app installed. What we could do:
QR payload is a url to a static webpage that leads to app stores and also shows the claimable value of the voucher.
That page also shows the private key as a Qar code or a smart link into the app

tasks:

implement js function reapVoucher that takes a secret uri string and a beneficiary account. Base it on the deprecated faucet
implement new QR type encointer-voucher as QrScanContext
implement new action trigger
MVP solution:

display confirmation dialog "do you want to transfer this voucher to your current account?"
call js wrapper for reapVoucher if yes

extensions:

show current balance of voucher before calling reap
allow to fund a voucher (only in develper mode?)

@clangenb please test against https://github.com/encointer/encointer-node/pull/228
one QR code example provided in the OP

As I have refactored the qr scanner stuff, we are more flexible and no longer limited with the number of lines etc. Each qr-code context can be configured independently. So I was wondering.
What do you think of the following format?
encointer-voucher
V1.0
//dgfiqw4525ng
sqm1v79dF6b
nctr-k
Name of issuer

Or is there another reason that you chose sqm1v79dF6b@nctr-k?

Actually, I think we should state the network also in invoice or claim QR codes. maybe even in contacts.
There is no specific reason to use the @ syntax. I just used it to stay in line with row definitions.
Separate lines are better I think.
If we have braking changes, please bump the QR-code version to V2.0


Gradle multi module project support

Hi,
when we open gradle multi module project problems in Problem tabs are not visible until we open specific class.
Reproduction is easy. setup simple gradle project with 2 modules and cause in both compile problem.
I didnt see any problems until open that class.
Same with project explorer or standard explorer - problems are visible there until class is open. Then these modules, classes change color.
Also gradle and language server do not detect any compile issues.

I am not sure if this issue is just being inherited from the netbeans java language server as I find that extension seems to flounder on multi module projects similarly.


feat(sms.senders): add column to display sender reason of reject

Ref: UXCT-99

@antleblanc I rebased the commit message.

@Steffy29 Some commits has been squashed on the develop branch. Could you please rebase yours? Thanks!

@antleblanc rebase done

@Steffy29 I'm not sure that you've rebase correctly.
Normally you should have these two commit only:
feat(sms.senders): add column to display sender reason of reject
build: add needed dependencies

OK. I tried many rebases yesterday and it was the best result I could have. I don't understand why I had all those commits in my PR.


SyntaxError

Showing some syntax error while running the latest code
2021-12-07T13:40:14.666267+00:00 heroku[worker.1]: State changed from crashed to starting
2021-12-07T13:40:18.167721+00:00 heroku[worker.1]: Starting process with command `python main.py`
2021-12-07T13:40:19.394530+00:00 heroku[worker.1]: State changed from starting to up
2021-12-07T13:40:19.850551+00:00 heroku[worker.1]: Process exited with status 1
2021-12-07T13:40:19.903699+00:00 heroku[worker.1]: State changed from up to crashed
2021-12-07T13:40:19.737423+00:00 app[worker.1]:   File "/app/main.py", line 78
2021-12-07T13:40:19.737444+00:00 app[worker.1]:     text=f"Error :- `{error}`\n\n`{'\n'.join(logs)}`",
2021-12-07T13:40:19.737584+00:00 app[worker.1]:                                                      ^
2021-12-07T13:40:19.737603+00:00 app[worker.1]: SyntaxError: f-string expression part cannot include a backslash

Fixed


Upgrade Alpine version for faster and more secure GO version

The current Dockerfile is using Alpine Linux 3.1 which was released in 2014-12-10 and bundles golang 1.3.3-r1.
Please use a newer Alpine Linux so Logspout can benefit from a faster and more secure golang version.

I'll accept a PR for this if you can make the changes.

Done, please review https://github.com/gliderlabs/logspout/pull/174


Translated BCL, reflection improvements.

Bootstrap/Core/Main.js was reordered for better readability, more classes skipped for Translated BCL.
$jsilcore.MemberInfoExternal is not required for PropertyInfo/FieldInfo.
MemberInfo mplements ICustomAttributeProvider (#804). For it I used a little bit hackish solution: please look JSIL.Initialize change.
DeclaringType proper implementation for Type.

Will work more on it - one test failed :)

Sorry for annoying, but should we discuss anything about it or should I make something more on this PR?

Thank you, with this merge it is much easier work with other Reflection improvements.


project template build failed

Hi, I'm following the tutorial and decided to build the project template, but it keeps failed to build and produce a lot of errors. I followed the exact steps in the documentation.
I guess the reason  it's generating so many errors is because there are multiple dependencies that are not compatible between my ros2 iron and ignition gazebo fortress?
Environment

OS Version: Ubuntu 22.04 LTS on arm64
Source or binary build?
binary build, the latest version

Description
Hi, I'm following the tutorial and decided to build the project template, but it keeps failed to build and produce a lot of errors. I followed the exact steps in the documentation.
I guess the reason  it's generating so many errors is because there are multiple dependencies that are not compatible between my ros2 iron and ignition gazebo fortress?
Steps to reproduce
I just follow the exact steps on the tutorial
Output
there are so many errors, I only include the most botton ones:
275 |   std::string* Mutable(Arena* arena);
|                ^~~~~~~
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:275:16: note:   candidate expects 1 argument, 2 provided
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:276:16: note: candidate: ‘std::string* google::protobuf::internal::ArenaStringPtr::Mutable(const google::protobuf::internal::LazyString&, google::protobuf::Arena*)’
276 |   std::string* Mutable(const LazyString& default_value, Arena* arena);
|                ^~~~~~~
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:276:42: note:   no known conversion for argument 1 from ‘const string*’ {aka ‘const std::cxx11::basic_string’} to ‘const google::protobuf::internal::LazyString&’
276 |   std::string Mutable(const LazyString& default_value, Arena* arena);
|                        ~~~~~~^
In file included from /usr/include/ignition/msgs8/ignition/msgs/MessageTypes.hh:226,
from /usr/include/ignition/msgs8/ignition/msgs/Utility.hh:34,
from /home/ubuntu/project_ws/src/ros_gz/ros_gz_sim/src/create.cpp:22:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h: In member function ‘std::string* ignition::msgs::WorldModify::release_cloned_uri()’:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h:591:29: error: no matching function for call to ‘google::protobuf::internal::ArenaStringPtr::Release(const string*, google::protobuf::Arena*)’
591 |   return cloned_uri.Release(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), GetArena());
|          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/ignition/msgs8/ignition/msgs/entity_factory.pb.h:25,
from /home/ubuntu/project_ws/src/ros_gz/ros_gz_sim/src/create.cpp:16:
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:304:35: note: candidate: ‘std::string* google::protobuf::internal::ArenaStringPtr::Release()’
304 |   PROTOBUF_NODISCARD std::string* Release();
|                                   ^~~~~~~
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:304:35: note:   candidate expects 0 arguments, 2 provided
In file included from /usr/include/ignition/msgs8/ignition/msgs/MessageTypes.hh:226,
from /usr/include/ignition/msgs8/ignition/msgs/Utility.hh:34,
from /home/ubuntu/project_ws/src/ros_gz/ros_gz_sim/src/create.cpp:22:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h: In member function ‘void ignition::msgs::WorldModify::set_allocated_cloned_uri(std::string*)’:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h:599:27: error: no matching function for call to ‘google::protobuf::internal::ArenaStringPtr::SetAllocated(const string*, std::string*&, google::protobuf::Arena*)’
599 |   cloned_uri.SetAllocated(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), cloned_uri,
|   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
600 |       GetArena());
|       ~~~~~~~~~~~
In file included from /usr/include/ignition/msgs8/ignition/msgs/entity_factory.pb.h:25,
from /home/ubuntu/project_ws/src/ros_gz/ros_gz_sim/src/create.cpp:16:
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:309:8: note: candidate: ‘void google::protobuf::internal::ArenaStringPtr::SetAllocated(std::string*, google::protobuf::Arena*)’
309 |   void SetAllocated(std::string* value, Arena* arena);
|        ^~~~~~~~~~~~
/home/ubuntu/anaconda3/include/google/protobuf/arenastring.h:309:8: note:   candidate expects 2 arguments, 3 provided
In file included from /usr/include/ignition/msgs8/ignition/msgs/MessageTypes.hh:226,
from /usr/include/ignition/msgs8/ignition/msgs/Utility.hh:34,
from /home/ubuntu/project_ws/src/ros_gz/ros_gz_sim/src/create.cpp:22:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h: In member function ‘std::string* ignition::msgs::WorldModify::unsafe_arena_release_cloned_uri()’:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h:607:22: error: ‘struct google::protobuf::internal::ArenaStringPtr’ has no member named ‘UnsafeArenaRelease’
607 |   return cloned_uri_.UnsafeArenaRelease(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(),
|                      ^~~~~~~~~~~~~~~~~~
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h: In member function ‘void ignition::msgs::WorldModify::unsafe_arena_set_allocated_cloned_uri(std::string*)’:
/usr/include/ignition/msgs8/ignition/msgs/world_modify.pb.h:618:15: error: ‘struct google::protobuf::internal::ArenaStringPtr’ has no member named ‘UnsafeArenaSetAllocated’
618 |   cloned_uri_.UnsafeArenaSetAllocated(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(),
|               ^~~~~~~~~~~~~~~~~~~~~~~
gmake[2]: *** [CMakeFiles/create.dir/build.make:76: CMakeFiles/create.dir/src/create.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:159: CMakeFiles/create.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Failed   <<< ros_gz_sim [5.32s, exited with code 2]
Aborted  <<< ros_ign_interfaces [2.04s]
Aborted  <<< ros_gz_bridge [14.0s]
Summary: 7 packages finished [17.7s]
1 package failed: ros_gz_sim
2 packages aborted: ros_gz_bridge ros_ign_interfaces
3 packages had stderr output: ros_gz_bridge ros_gz_example_gazebo ros_gz_sim
8 packages not processed

Same issue here with dependencies.  Has this been fixed?
OS Version: Ubuntu 22.04 LTS
ROS: Humble
Gazebo: Fortress
Followed template instructions and removed/reinstalled gazebo, but there are dependencies that must not be in the default packages:  for example "gz-plugin2"

same here, got errors when building
**CMake Error at CMakeLists.txt:13 (find_package):
By not providing "Findgz-cmake3.cmake" in CMAKE_MODULE_PATH this project
has asked CMake to find a package configuration file provided by
"gz-cmake3", but CMake did not find one.
Could not find a package configuration file provided by "gz-cmake3" with
any of the following names:
gz-cmake3Config.cmake
gz-cmake3-config.cmake

Add the installation prefix of "gz-cmake3" to CMAKE_PREFIX_PATH or set
"gz-cmake3_DIR" to a directory containing one of the above files.  If
"gz-cmake3" provides a separate development package or SDK, be sure it has
been installed.
**

same here, got errors when building **CMake Error at CMakeLists.txt:13 (find_package): By not providing "Findgz-cmake3.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "gz-cmake3", but CMake did not find one.
Could not find a package configuration file provided by "gz-cmake3" with any of the following names:
gz-cmake3Config.cmake
gz-cmake3-config.cmake

Add the installation prefix of "gz-cmake3" to CMAKE_PREFIX_PATH or set "gz-cmake3_DIR" to a directory containing one of the above files. If "gz-cmake3" provides a separate development package or SDK, be sure it has been installed. **

I had this error before it is because you don't have required dependency on your system, you have to install gz-cmake package manually, check https://gazebosim.org/libs/cmake/

Same issue here. I'm running a docker file (FROM osrf/ros:jazzy-desktop-full) and I'm trying to build the template project, but it fails with the following error:
--- stderr: ros_gz_example_gazebo
CMake Error at CMakeLists.txt:26 (find_package):
  By not providing "Findgz-sim7.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "gz-sim7", but
  CMake did not find one.

  Could not find a package configuration file provided by "gz-sim7" with any
  of the following names:
    gz-sim7Config.cmake
    gz-sim7-config.cmake
  Add the installation prefix of "gz-sim7" to CMAKE_PREFIX_PATH or set
  "gz-sim7_DIR" to a directory containing one of the above files.  If
  "gz-sim7" provides a separate development package or SDK, be sure it has
  been installed.

---
Failed   <<< ros_gz_example_gazebo [0.42s, exited with code 1]

Relates to #37


attribute "title" for option element

attribute "title" for option element
Example:
<select>
  <option value="1" title="Germany">DE</option>
  <option value="2" title="United Kingdom">GB</option>
  <option value="3" title="United States">US</option>
</select>

http://jsfiddle.net/Nt5xv/

I don't see a reason not to support this by default in select2, since title works for vanilla <select> in most browsers.
In any case,
function(item) {
    return $('<div>', {title: item.element[0].title}).text(item.text);
}

works for formatResult/formatSelection.


Should optimize submission process.

I was looking at the appstats for a submission, and I noticed that a Participant.add_role call issues a query, which is expensive. We should cache this.
Also, the put call in there can be turned into a put_async call, as long as we make sure the top level thing in api.py has the @ndb.toplevel decorator.
Also, we should cache lookup_assignments_by_name in api.py.

Here are the docs for GAE memcache. It says that the max size of values in memcache is a little less that 1 MB. Not sure if the list of assignments will fit in there... Maybe we should only store certain things we care about in memcache?


picoquic-create not found

Here I am trying to build wrk-quic but it shows there is no picoquic-create in picoquic-core module

Would you have an old picoquic maybe? I see it's available in the latest picoquic. Can you check the picoquic-core library with objdump?

I tried building picoquic from here https://github.com/private-octopus/picoquic

I think there is picoquic-core lib but it shows PIcoquic not found....

Is this repository checked out in wrk2?
Maybe something was updated in picoquic, there's no versioning so I'll try to find a working commit.

Picotls added a new library used in picoquic. As they refuse to create a proper install system like pkgconfig, I can't automatize the libraries to be added...
Anyway, it's fixed.


Files in the root of the shared folder is not being included in scans

Hi,
I've come across the following, that i think is a bug.
If i create a share on a Windows server (tested on 2019 and 2022) and place a file in the root of the folder being shared, it is not included in a scan.
Tried with both visible and hidden shares, no difference in the results.
A folder in the root of the shared folder is being included in the scan as a child directory.
/Rasmus

Thanks for raising @rgorme, let me see if I can locally reproduce

Hi,
I've tried to reproduce this but cant.  I have created a share with only one file in it, and no subfolders, and the file is detected fine and stored in the CSV.
SHARE>
---> FILE.txt
Could it be the permissions on the file are preventing SMBeagle from seeing it?

Output from a run with following command line is shown below.
.\SMBeagle.exe -c C:\Temp\SMBeagle\shares.csv -D -h 10.x.x.x -verbose; Import-Csv -Path "C:\Temp\SMBeagle\shares.csv" -Delimiter ","
Content of folder shares as both Testing and Testing$ shares, is a plain text file called test.txt.
As the same user running the command, i can read and edit the file using both shares.
    ____              __   _____                      _ __
   / __ \__  ______  / /__/ ___/___  _______  _______(_) /___  __
  / /_/ / / / / __ \/ //_/\__ \/ _ \/ ___/ / / / ___/ / __/ / / /
 / ____/ /_/ / / / / ,<  ___/ /  __/ /__/ /_/ / /  / / /_/ /_/ /
/_/    \__,_/_/ /_/_/|_|/____/\___/\___/\__,_/_/  /_/\__/\__, /
                                       PRESENTS         /____/

                     -- SMBeagle v2.1.0 --

Skipping network discovery due to -D switch...
Skipping filtering as network discovery disabled...
Processing manual networks and addresses...
added host '10.x.x.x'
Probing hosts and scanning networks for SMB port 445...
scanning is complete and we have 1 hosts with reachable SMB services
reachable hosts:
10.x.x.x
Probing SMB services for accessible shares...
probing is complete and we have 1 hosts with accessible shares
reachabled hosts with accessible SMB shares:
10.x.x.x
accessible SMB shares:
\10.x.x.x\admin$
\10.x.x.x\c$
\10.x.x.x\h$
\10.x.x.x\ipc$
\10.x.x.x\testing
\10.x.x.x\testing$\
Enumerating accessible shares, this can be slow...
6a. Enumerating all subdirectories for known paths
Enumerating all subdirectories for '\10.x.x.x\admin$'
Enumerating all subdirectories for '\10.x.x.x\c$'
Enumerating all subdirectories for '\10.x.x.x\h$'
Enumerating all subdirectories for '\10.x.x.x\ipc$'
Enumerating all subdirectories for '\10.x.x.x\testing'
Enumerating all subdirectories for '\10.x.x.x\testing$'
6b. Splitting large directories to optimise caching and to batch output
6c. Enumerating files in directories
Found 0 child directories and 0 files in '\10.x.x.x\admin$'
Found 0 child directories and 0 files in '\10.x.x.x\c$'
Found 0 child directories and 0 files in '\10.x.x.x\h$'
Found 0 child directories and 0 files in '\10.x.x.x\ipc$'
Found 0 child directories and 0 files in '\10.x.x.x\testing'
Found 0 child directories and 0 files in '\10.x.x.x\testing$'
file enumeration complete, 0 files identified
Completing the writes to CSV or elasticsearch (or both)
-- AUDIT COMPLETE --

Thanks for the output.  I'm still trying to reproduce here.
Could you try providing the username and password with -u and -p (and -d for domain if you need it).  This switches to a different module for SMB enumeration.
I'll try and reproduce what you are seeing with the windows native SMB support you are using.

Okay I've reproduced this now and will dig into it.  It does look to only apply on the windows native scans, so providing a -u  and -p  should help  for now :)

Great :-)
I will give it a try using -u and -p.

Hi, this is should now be fixed in the latest release.
Can you test?  No -u/-p  needed.
https://github.com/punk-security/SMBeagle/releases/tag/2.2.1

Hi,
I'm only able to test on friday this week.
I will get back to you.
Rasmus

Hi @SimonGurney,
I've just tested and it works perfectly!
Thx alot :-)
Rasmus


experiment

Doing a forced merge to test the re-release action. No effect on build outputs or product source.


Automated cherry pick of #4520: baremetal: fix megaraid check jbod always disabled

Cherry pick of #4520 on release/2.12.
#4520: baremetal: fix megaraid check jbod always disabled

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.12@4422faf). Click here to learn what that means.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             release/2.12   #4524   +/-   ##
==============================================
  Coverage                ?   6.82%
==============================================
  Files                   ?     501
  Lines                   ?   85462
  Branches                ?       0
==============================================
  Hits                    ?    5837
  Misses                  ?   79131
  Partials                ?     494

Flag
Coverage Δ

#aFlag
6.82% <ø> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4422faf...02982f7. Read the comment docs.


Requesting expanded data for a product without expanded data causes empty set results

http://data.neonscience.org/api/v0/data/DP1.10072.001/TALL/2014-09?package=expanded returns
{
"status": 200,
"data": {
"files": [],
"productCode": "DP1.10072.001",
"urls": [],
"month": "2014-09",
"siteCode": "TALL"
}
}
but should probably return the basic data instead.

I think that this should not return the basic package, as this could be confusing to the end user. Instead, a message could be returned - "An expanded package does not exist for this data product. If you would like the basic package, use the following URL: [correct URL for basic package]".


[ES 7.3.1] Error: Skipping non-plugin directory at /usr/share/kibana/plugins/enhanced-table

Hey Fabien,
Thanks for the plugin.
I seem to have trouble updating the plugin from ES 7.3.0 to ES 7.3.1 (with your plugin version staying at 1.4.3):
# Remove previous installation
/usr/share/kibana/bin/kibana-plugin remove enhanced-table --allow-root

# Install the new version
/usr/share/kibana/bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip --allow-root

# Restart Kibana to get the new plugin
systemctl restart kibana

Everything good thus far.
package.json gives me this:
{
  "name": "enhanced-table",
  "version": "1.4.3",
  "description": "This visualization plugin is like a Data Table, but with enhanced features like computed columns and filter bar",
  "main": "index.js",
  "kibana": {
    "version": "7.3.1"
  },
  "dependencies": {
    "expr-eval": "^1.2.2"
  },
  "build": {
    "git": {
      "count": "111",
      "sha": "79df487",
      "date": "Thu, 8 Aug 2019 21:37:31 +0200"
    },
    "date": "Thu Aug 08 2019 22:03:20 GMT+0200 (GMT+02:00)"
  }
}

In the Kibana log file however, I get this message: Skipping non-plugin directory at /usr/share/kibana/plugins/enhanced-table
Can you help?
Thanks

This is weird...
I invite you to do these actions:

stop Kibana
remove $KIBANA_HOME/optimize folder
start Kibana

No dice sorry. It still comes with the same error message:
{"type":"log","@timestamp":"2019-08-26T12:59:38Z","tags":["plugin","warning"],"pid":24683,"path":"/usr/share/kibana/plugins/enhanced-table","message":"Skipping non-plugin directory at /usr/share/kibana/plugins/enhanced-table"}

Did you try to launch Kibana UI in your browser and check that plugin is available?

Yep, I get this:

...just like a new upgrade without the plugin :(

I've just tried to purge my entire Kibana installation:
apt purge kibana

..leaving only my raw kibana data folder back (/var/lib/kibana/). Reinstalled and reconfigured, and then installed the plugin. Still no dice.
{"type":"log","@timestamp":"2019-08-27T05:25:40Z","tags":["info","plugins-system"],"pid":24465,"message":"Starting [1] plugins: [translations]"}

Very strange...
Could you :

stop Kibana
remove optimize and plugins/enhanced-table folders
install enhanced-table plugin
start Kibana

Hey Fabien,
I figured out something. I switched to the local kibana user (using su kibana -s /bin/bash), and tried to install the plugin, and it came with this "new" (well uncaught) error:
kibana@kibana1:/usr/share/kibana$ bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip
Attempting to transfer from https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip
Transferring 52125 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Plugin installation was unsuccessful due to error "EACCES: permission denied, open '/usr/share/kibana/plugins/.plugin.installing/package.json'"

That file doesn't exists at all, so I tried to create that file..
kibana@kibana1:/usr/share/kibana$ mkdir /usr/share/kibana/plugins/.plugin.installing/
kibana@kibana1:/usr/share/kibana$ touch /usr/share/kibana/plugins/.plugin.installing/package.json

And re-ran the install:
kibana@kibana1:/usr/share/kibana$ bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip
Found previous install attempt. Deleting...
Attempting to transfer from https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip
Transferring 52125 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Plugin installation was unsuccessful due to error "EACCES: permission denied, open '/usr/share/kibana/plugins/.plugin.installing/package.json'"

The kibana user has rwx (775) all the way through /usr/share/kibana/plugins/ and the optimize folder.

Create the /usr/share/kibana/plugins/enhanced-table folder manually, and then installing the plugin again, gives the "plugin already installed" error. HOWEVER, it leaves the /usr/share/kibana/plugins/.plugin.installing/ folder. Look at these permessions:
root@kibana1.clm-test.jppol.net:~# ls -alt /usr/share/kibana/plugins/.plugin.installing/
total 12
drwxrwxr-x 4 kibana kibana   76 Aug 27 08:44 .
drwxrwxr-x 4 kibana kibana 4096 Aug 27 08:44 public
-----w---- 1 kibana kibana  490 Aug 27 08:44 package.json
-----w---- 1 kibana kibana  479 Aug 27 08:44 index.js
drwxrwxr-x 3 kibana kibana   23 Aug 27 08:44 node_modules
drwxrwxr-x 4 kibana kibana   54 Aug 27 08:44 ..

There is only write access to package.json and index.js - are you setting those permissions? Or how are they set?
Thanks for the help Fabien.

Well, here it is clearly a Linux user permission problem.
You can install the plugin with whatever user who have write access to plugins folder.
Then, make sure that user who runs Kibana has read access to whole Kibana folder, including all enhanced-table plugin folders and files.

Hey @fbaligand,
Indeed it is.
Doing it this way works:
# Install
kibana@kibana1:/usr/share/kibana$ bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip

# Correct permissions
root@kibana1:/usr/share/kibana# chown kibana:kibana plugins/ -R
root@kibana1:/usr/share/kibana# chmod 755 plugins/ -R

# Restart Kibana
root@kibana1:/# systemctl restart kibana

HOWEVER, I tried to download https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.0.zip (for ES 7.3.0), and it gives these DIFFERENT permissions, which work for the Kibana users, without the above workaround:
# Create folder to keep the .plugin.installing folder
kibana@kibana2:/usr/share/kibana$ mkdir plugins/enhanced-table

# Download previous release
kibana@kibana2:/usr/share/kibana$ bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.0.zip

# Get permissions
kibana@kibana2:/usr/share/kibana$ ls -alt plugins/.plugin.installing/
total 12
drwxrwxr-x 4 kibana kibana   76 Aug 27 13:25 .
drwxrwxr-x 4 kibana kibana 4096 Aug 27 13:25 public
drwxrwxr-x 3 kibana kibana   23 Aug 27 13:25 node_modules
-rw-rw-r-- 1 kibana kibana  490 Aug 27 13:25 package.json
-rw-rw-r-- 1 kibana kibana  479 Aug 27 13:25 index.js
drwxrwxr-x 4 kibana kibana   54 Aug 27 13:25 ..

I think there is a permission problem in the 7.3.1 release, since it works with the previous release.

Thanks for your feedback.
First, happy to see it finally works.
Then, indeed, package for Kibana 7.3.1 has been packaged using a different way from Kibana 7.3.0. It is probably the reason of the difference.
I will try to fix it.

Hi @magnuslarsen ,
I just packaged and published a new package for Kibana  :
https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.4.3/enhanced-table-1.4.3_7.3.1.zip
Could you try to install it and tell me if works fine without workaround?

Hey @fbaligand,
Absolutely, and it works!
Thanks for the prompt help and effort. And thanks for the plugin :)

Thanks for your feedback!
And happy to see that you appreciate the plugin :)

Hey @fbaligand
I tried to download the new package this morning on my production system, and it seems that the node_modules and public folders are completely empty, causing the plugin to crash.
I tried to download and open the ZIP file in my browser, and indeed, that is empty too.

Very sad to hear that.
Sorry for this partial package, I will fix it today.

Hi,
I just published 7.3.1 package.
This time, I use exactly the same way than for 7.3.0 package.
It should work as expected.
Sorry for this issue...

No problem!
Let me know if you want to test the new deployment system in the future :+1: - I update our test cluster as soon as your plugin is released for the new Elastic version, so it shouldn't be a problem :)
Also i can confirm that it works completely now!

Thanks for your help!
Community help is always much appreciated!
If you didn’t do it, you can subscribe to new releases in the github repository.
Like this, you are notified by mail each time a new release is published.
That said, if I published a new package for an existing release, you shouldn’t be notified.
And so, if you see a bug on a fresh new release, don’t hesitate to open an issue!


docker_build.Image outputs the manifest sha

What happened?
I'm having troubles setting up a simple image build / container run using pulumi_docker_build. The container step fails because it seems that the image outputs the manifest sha instead of the config sha.
Example
bash
 docker image rm test-image || true # cleanup
 pulumi destroy -s dev --yes || true # cleanup

 pulumi up -s dev --diff --yes # this fails with the following

output
[...]

#8 exporting manifest sha256:58f5eef691acc30cc1c9790cca76899865ee4c1ed4ffb7d18bf3e948fd4cbba5 done
#8 exporting config sha256:8ff66f8e9f159a99c5f34c6368e4a6a47205c343c345f8abd82edce3cef4a226 done

[...]

error: 1 error occurred:
	* Unable to create container with image sha256:58f5eef691acc30cc1c9790cca76899865ee4c1ed4ffb7d18bf3e948fd4cbba5: unable to pull image sha256:58f5eef691acc30cc1c9790cca76899865ee4c1ed4ffb7d18bf3e948fd4cbba5: error pulling image sha256:58f5eef691acc30cc1c9790cca76899865ee4c1ed4ffb7d18bf3e948fd4cbba5: Error response from daemon: pull access denied for sha256, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

    --outputs:--
    context_hash: "d6353da46ae0c3d9d35848b9386745132e9f8cd7628954ca4409c2bd9b8daca5"
    digest      : "sha256:58f5eef691acc30cc1c9790cca76899865ee4c1ed4ffb7d18bf3e948fd4cbba5"
    id          : "sha256:58f5eef691acc30cc1c9790cca76899865ee4c1ed4ffb7d18bf3e948fd4cbba5"
    ref         : "test-image"
error: update failed
Resources:
    + 3 created

Duration: 22s

You can see that in docker image the sha shown is the config sha and not the manifest sha:
docker image ls | head -n 4
REPOSITORY                     TAG                                                       IMAGE ID       CREATED          SIZE
test-image                     latest                                                    8ff66f8e9f15   24 minutes ago   922MB

My setup
 tree -L1 .
.
├── Dockerfile
├── Pulumi.yaml
├── __main__.py
├── __pycache__
├── copied
├── ignored
├── requirements.txt
└── venv

cat Dockerfile
FROM python:3.11-buster

RUN pip install networkx

COPY ./copied /copied

cat Pulumi.yaml
name: test_project
runtime:
  name: python
  options:
    toolchain: pip
    virtualenv: venv
description: A minimal Python Pulumi program
config:
  pulumi:tags:
    value:
      pulumi:template: python

cat __main__.py
"""A Python Pulumi program"""

import pulumi
import pulumi_docker as docker
import pulumi_docker_build as docker_build

provider = docker_build.Provider("test-provider", host="unix:///var/run/docker.sock")

image = docker_build.Image(
    "test-image",
    context=dict(location="."),
    dockerfile=dict(location="./Dockerfile"),
    platforms=[docker_build.Platform.LINUX_AMD64],
    tags=["test-image"],
    push=False,
    opts=pulumi.ResourceOptions(provider=provider),
    exports=[{"docker": {'tar': True}}],
)
pulumi.export("digest", image.digest)
pulumi.export("ref", image.ref)
pulumi.export("id", image.id)
pulumi.export("context_hash", image.context_hash)

backend = docker.Container(
    "container",
    name=f"container",
    image=image.digest,
)

Output of pulumi about
CLI
Version      3.132.0
Go Version   go1.23.1
Go Compiler  gc

Plugins
KIND      NAME          VERSION
resource  docker        4.5.5
resource  docker-build  0.0.6
language  python        unknown

Host
OS       darwin
Version  14.6.1
Arch     x86_64

This project is written in python: executable='/Users/nico/test/077/venv/bin/python' version='3.12.5'

Backend
Name           pulumi.com
URL            https://app.pulumi.com/Ngoguey42
User           Ngoguey42
Organizations  Ngoguey42
Token type     personal

Dependencies:
NAME                 VERSION
pip                  24.2
pulumi_docker        4.5.5
pulumi_docker_build  0.0.6
setuptools           75.1.0
wheel                0.44.0

Pulumi locates its logs in /var/folders/50/ccj8r__12yz02g03vhq93r7c0000gn/T/ by default
warning: Failed to get information about the current stack: No current stack

Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

digest contains a hash of the exported manifest, but this isn't the same thing as the local (intermediate) "Image ID" shown in the CLI. I'd recommend using image=image.ref for your container.


[rfr] Prevent changing attached DraftRegistration file

Purpose
Prevent users from changing an attached file when prereg form is in progress.
Changes

Any file attached to a DraftReg becomes "checked-out" when the preregistration form is submitted.
The files become "checked-in" when the registration is approved or sent back as rejected.
If the user tries to check-in an attached file before the DraftReg is approved, they receive a warning that this action will cancel their submission. If they continue, any DraftReg with that file attached will be rejected and all files attached to it become checked-in.

Side effects

Expose /reject/ route on front end. I don't like this so I'd like suggestions on the best way to handle it.
Checking in an attached file can do a lot if the user doesn't read the warning.

Ticket
https://openscience.atlassian.net/browse/OSF-5585
Depends on #5280

Investigations—for @whit537

Punchlist—for @barbour-em

(Reposting from Jira since I'm not sure which channel is best ...)

users are experiencing registration failure when they change a file attached to a preregistration form while the registration is under review (in progress).

@barbour-em Could I get steps/expected/actual for this? I'm trying to reproduce the original bug so I can verify that this PR fixes the bug. Thanks! :-)

Here's where I'm at so far:

create a new project
add a file ({{foo.txt}}, contents Cheese!)
Registrations > [ New registration ] > open-ended [ Create draft ] > "blah text" [ Preview for submission ]

What I was able to do:

Create project
Add a file foo.txt
Registration > New Registration > Prereg Challenge > Attach foo.txt to at least one question that allows file attachments (e.g. Design Plan->Study Design)
Complete Prereg Registration > Preview for submission > Submit for review
Edit foo.txt while the registration is pending approval
Approve registration via admin interface

Prereg Challenge

Gotcha ... lessee ...

Eternal spinner on a file upload widget, investigating ...

Have you pulled from dev since #5280 was merged?
https://github.com/CenterForOpenScience/osf.io/blob/e731c4c1598f2118195528f8b818bd99dfc7dcad/website/static/js/registrationEditorExtensions.js#L170

No! Good idea. :-)

Hrm ... but I'm actually on develop. That's maybe what you meant, though, ya?

Oops, yes that is what I meant

Travis failures seem odd; all the tests pass locally. The log also includes this gem:

ERROR: Failure: MultipleResultsFound (Query for find_one must return exactly one result; returned 1)

Alright, I'm able to reproduce the bug on develop at 643bb27.

Oh wait ... looks like it's actually a perms error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 63, in __call__
    return self.application(environ, start_response)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 177, in __call__
    response = self.get_response(request)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 230, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 149, in get_response
    response = self.process_exception_by_middleware(e, request)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 147, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/Users/whit537/workbench/cos/osf.io/env/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/Users/whit537/workbench/cos/osf.io/admin/pre_reg/views.py", line 123, in approve_draft
    draft.approve(user)
  File "/Users/whit537/workbench/cos/osf.io/website/project/model.py", line 4681, in approve
    self.approval.approve(user)
  File "/Users/whit537/workbench/cos/osf.io/website/project/model.py", line 4476, in approve
    raise PermissionsError("This user does not have permission to approve this draft.")
PermissionsError: This user does not have permission to approve this draft.

I did go through the prereg instructions ...

Looks like there's a setting for this, which is prereg_admin by default. The docs specify prereg_group instead.

I added an empty prereg_admin group and added my admin user to that group. I still get the same permissions error. Interestingly, system_tags is empty:
(Pdb) l
4471                    'TODO: add a generic email template for registration approvals'
4472                )
4473
4474        def approve(self, user):
4475            import pdb; pdb.set_trace()
4476 ->         if settings.PREREG_ADMIN_TAG not in user.system_tags:
4477                raise PermissionsError("This user does not have permission to approve this draft.")
4478            self.state = Sanction.APPROVED
4479            self._on_complete(user)
4480
4481        def reject(self, user):
(Pdb) user
<EMAIL_ADDRESS>with id u'ahn93'>
(Pdb) user.system_tags
[]
(Pdb)

Approve registration via admin interface

@barbour-em What are the steps to do this? I've tried following these docs and am hitting the permissions error described above.

@whit537 Interestingly I'm having the permissions problem now too, I could approve before. I guess for now you could do
(in inv shell)
In[1]: from website.settings import PREREG_ADMIN_TAG
In[2]: user = models.User.load('ahn93')
In[3]: user.system_tags.append(PREREG_ADMIN_TAG)
In[4]: user.save()
Out[4]: {'system_tags'}
In[5]: commit()
Transaction committed.
New transaction opened

This fixed the issue for me

Alright, I've manually added PREREG_ADMIN_TAG to my user.system_tags. However, I'm not able to produce a failure from Step 6: Approve registration via admin interface. I'm going to try again, maybe I missed or messed up a step or something ...

Steps

Start with develop at 643bb27
Reset Mongo for osf.io.
Reset SQLite for admin.
Create a new osf.io account.
Create a new admin account.
Configure for prereg work, part 1
Configure for prereg work, part 2
Create a project.
Add a foo.txt file.
Registrations > New registration > Prereg Challenge
asdf for all required textareas
...

I'm back at https://github.com/CenterForOpenScience/osf.io/pull/5464#issuecomment-210552553 for all eight file widgets on the prereg form.

Hrm ... "Save as Draft" and then "Edit" seems to reset the file upload widgets to a working state ... until the fifth one, which is Design Plan > Study Design. After failing to load that widget on the page, I go back to the four previous file upload widgets and they're all now broken.
If I navigate to "Design Plan" and reload the page, the 5th file upload widget works.

If I navigate to "Variables" (the previous pseudo-page) and back, the 5th file upload widget is broken again.

Alright, whatever. This is all on develop, so out of scope here. Just trying to figure out how to work around this bug so I can test here ...

Steps

Start with develop at 643bb27
Reset Mongo for osf.io.
Reset SQLite for admin.
Create a new osf.io account.
Create a new admin account.
Configure for prereg work (part 1).
Configure for prereg work (part 2).
Create a project.
Add a foo.txt file, contents Cheese!\n (8 bytes).
Registrations > New registration > Prereg Challenge
Enter asdf for all required textareas.
Use workaround to select foo.txt for Design Plan > Study design.

Preview for submission.
Submit for review.
"Make registration public immediately"
Check admin, prereg is "Pending Approval":
Clicking "Approve" at this point should result in green "Approved". But ... !
Files > OSF Storage > Upload > foo.txt, contents Bread!\n (7 bytes):
Back to admin ...
Approve > Approve:

Expected
Failure to approve. A 500? A red "Failed" status?

Approved.

@whit537 These are the steps that I have used before to cause this issue, I re-ran them this morning with the same results as you. It should have given a 500. Checking the database the registration seems to point to the updated file.
Maybe the bug was fixed elsewhere? Or the error was a result of something else that has been fixed between now and when I started work on this?
Either way, I'm not sure what to do with this PR if we can't reproduce. What are your thoughts?

@barbour-em I would say spend up to an hour tracking down why the issue is resolved. If we can't answer the question in a reasonable amount of time then we should just let it go. Either way it looks like we'll be able to retract this PR.


Performance feedback

What happened?
rules_swc is great, but I think the performance of this ruleset could be improved by some tweaks to the action graph it creates. Presently, one action is created per file, which creates immense overhead. Doing a cold build of a large codebase takes 30+ mins when SWC unencumbered could probably eat through it in less than a minute. This appears to be Bazel overhead from spawning so many actions, and them consequently being queued up. For (a) performance and (b) because it didn't appear to add much value, I disabled sandboxing in .bazelrc for SWCTranspile, but needless to say, sandboxing on top is going to further inflate the time taken. I imagine one-action-per-file is also costly from a remote cache / RBE perspective.
After a cold build, incremental build time is superb, but I think it would be just as fast with, say, 5-10 files per action as 1. Perhaps an int attr could be added that specifies how many files will be inputs to each action, with a default that is greater than 1 but not too large either, to enable some economy of scale..
Version
~
How to reproduce
No response
Any other information?
No response
Fund our work

[ ] Sponsor our open source work by donating a bug bounty

Hey @matthewjh , the next release removes the NodeJS spawn for each swc action which I think was the majority of the time. Now we spawn the bare rust swcx cli which should have excellent cold-start performance.
Could you check whether it seems improved for your use case?

Thanks @alexeagle. Performance seems much improved by that change. Will close this and re-open if I spot anything else.


Rework Reports (new history)

Rework of #9722 after #9749 had been merged

Thanks for the initial suggestions, @larsoner! Some good catches. I'll try to address all of them and ping you.

Rendered docs:
Tutorial
API docs

@wmvanvliet @rob-luke If you're brave and curious, I'd like to hear how this works for you 🤞

Sorry @hoechenberger I won't have time to try this locally in the next week or so. But I checked out the tutorial and it looks great. The pages render very nicely and you still have the custom plot section, so that works for me. Cool stuff. Ill check back when I have a chance to code again.

Latest doc builds:
Tutorial
API docs

I'll push a commit to fix/modernize @deprecated, okay for me to git merge upstream/main while I'm at it to fix the conflict?

@larsoner yes absolutely, feel free to do anything except for force-pushes :)

I found some bugs looking at reports I generated using old/deprecated methods. I suggest you fix these first, then once they all work, I can try moving over to new functions, and we can fix any bugs there. Finally I'll try using all old functions one more time, and if both deprecated + new functions render everything properly then hopefully we're good to go!
So to start, using old/deprecated functions -- I hope you can replicate locally using something simple (or immediately see what the problem can be from the images), but if not, I can whittle things down to some minimal example code:

Header occlusion bug -- first render of a custom report:

Resizing the window fixes it:

BEMs not resized properly to fit in frame:

Headers overlapping TOC column:

Sliders don't actually change the shown images in a custom slider I made:

.

Ouch.
None of these should happen.
Which browser are you using?

Updated deprecation links nicely in our docs I think:

Chrome on Linux. Looking at the console, this seems problematic:
DevTools failed to load source map: Could not load content for file:///home/larsoner/mne_data/mnefun/examples/funloc/subj_01/bootstrap.bundle.min.js.map: System error: net::ERR_FILE_NOT_FOUND
DevTools failed to load source map: Could not load content for file:///home/larsoner/mne_data/mnefun/examples/funloc/subj_01/bootstrap.min.css.map: System error: net::ERR_FILE_NOT_FOUND

Perhaps this is causing the sliders not to work at least? Not sure why the path would be messed up here...

Thank you! Which version of chrome is this?

Thanks @larsoner, will ping you when it's ready

Okay I can replicate all the problems above with this:

import os.path as op
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

import mne

data_path = mne.datasets.testing.data_path() + '/SSS/'
fname = data_path + 'test_move_anon_raw.fif'
struc = 'sample'
subjects_dir = data_path + '../subjects'
trans = data_path + '../MEG/sample/sample_audvis_trunc-trans.fif'
fname_inv = data_path + '../MEG/sample/sample_audvis_trunc-meg-eeg-oct-4-meg-inv.fif'

raw = mne.io.read_raw_fif(fname, allow_maxshield='yes').load_data()
raw.rename_channels(lambda x: f'{x[:3]} {x[3:]}' if 'MEG' in x else x)
fname_pos = data_path + 'test_move_anon_raw.pos'
style = {'axes.spines.right': 'off', 'axes.spines.top': 'off',
         'axes.grid': True}
matplotlib.use('Agg', force=True)
plt.rcParams.update(style)
report = mne.Report(verbose=False)

# %%
report.add_htmls_to_section(
    '
Blah blah blah
',
    captions='Behavioral', section='Behavioral')

# %%
events = mne.make_fixed_length_events(raw)
epochs = mne.Epochs(raw, events, preload=True)
section = 'Drop log'
figs = [epochs.plot_drop_log()]
captions = [repr(epochs)]  # <--- Does not render properly!
report.add_figs_to_section(figs, captions, section, image_format='svg')  # <--- image_format='svg' BREAKS THINGS!
evoked = epochs.average()

# %%
section = 'BEM'
caption = '%s: %s' % (section, struc)
report.add_bem_to_section(
    struc, caption, section, decim=10, n_jobs=1,
    subjects_dir=subjects_dir)

report.save('report.html', overwrite=True)

Looks like two bugs:

SVG support broken
Sanitizing bugs for HTML (repr(epochs) as a caption is problematic)

@larsoner Oh, super good catch! I think I only really tested with PNGs!
I'm currently a little bit lost at to which problems still persist, I guess I'll be in touch on Discord! Talk to you soon

All of the problems enumerated 2h ago persist for me, probably due to the causes I just mentioned, and you should be able to replicate locally by running the example script I posted above

(I pushed commits to fix some issues orthogonal to these, namely the deprecation inadequacy and a BEM axis labeling bug)

@larsoner Thanks, I'll get to work on this now!

@hoechenberger Just played around with the new report a little. Opening an existing report and adding/replacing figures in it works great!

Remaining CI complaints will go away once #9805 is merged

CI error seems to due to time limits on Azure, but unrelated to the changes in this PR
@larsoner lmk if you're happy or if you experience any remaining issues!

Rendered tutorial: https://35276-1301584-gh.circle-artifacts.com/0/dev/auto_tutorials/intro/70_report.html#sphx-glr-auto-tutorials-intro-70-report-py
API docs: https://35276-1301584-gh.circle-artifacts.com/0/dev/generated/mne.Report.html?highlight=report#mne.Report

Test time is still a problem (maybe MRI rendering, maybe something else):
74.13s call     mne/report/tests/test_report.py::test_full_report

The only report-related ones that show up in a different PR's recent build are:
23.52s call     mne/report/tests/test_report.py::test_render_report[mayavi]
23.08s call     mne/report/tests/test_report.py::test_render_report[pyvistaqt]

@larsoner Ouch, good catch. I managed to cut down the test time drastically on my local machine, and I'm surprised tests still take so long on Circle. Back to the profiler, I suppose!

3D plotting is much slower there, my guess is it's the new alignment plotting function. One option is to make it so the view/size there is very small so there is less to render

On Linux you can test this using xvfb-run because it will use software rendering like the CIs -- want me to try locally?

Locally there is at least some slowdown from software rendering, so I think it probably is the 3D rendered plot_alignment slowing things down:
$ pytest mne/report
25.12s call     mne/report/tests/test_report.py::test_render_report[pyvistaqt]
15.49s call     mne/report/tests/test_report.py::test_full_report
...
$ xvfb-run pytest mne/report
23.49s call     mne/report/tests/test_report.py::test_render_report[pyvistaqt]
21.96s call     mne/report/tests/test_report.py::test_full_report

And with if get_3d_backend() is not None and False: in test_report.py:656 so that the 3D stuff is skipped:
23.22s call     mne/report/tests/test_report.py::test_render_report[pyvistaqt]
9.85s call     mne/report/tests/test_report.py::test_full_report

So locally I would target the four calls in those to speed them up somehow. Making the render window smaller is one way.

... actually hold on, I think we have a regression with MNE_3D_BACKEND_TESTING where we automagically made it so that these things were faster. Let me see if I can fix it quickly.

Oh yes, I think you might be in the right track here!!

Nope, no regression -- really the slowdown here is caused because the renderer fixture is not used. But We can easily get around it by using the same actor-invisible trick. I've pushed a commit that locally fixes things, timings for xvfb-run:
22.97s call     mne/report/tests/test_report.py::test_render_report[pyvistaqt]
12.97s call     mne/report/tests/test_report.py::test_full_report

I'll push and we'll see how the CIs do. But really these are both still too slow, this is what I see on main using the same xvfb-run call:
9.07s call     mne/report/tests/test_report.py::test_render_report[pyvistaqt]
2.49s call     mne/report/tests/test_report.py::test_render_mri[pyvistaqt]

But really these are both still too slow, this is what I see on main using the same xvfb-run call:

But keep in mind that we're actually plotting much more now than we used to, so a bit of slowdown is to be expected. But I agree, the current slowdown is too much.

Thx @hoechenberger !


[Feature] Update Iceberg to 1.0 and add REST catalog support

What type of PR is this：

[ ] BugFix
[X] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Many Iceberg-based systems are moving from the Hive metastore to the recently added REST catalog. This PR updates Iceberg dependencies to version 1.0.0, and adds support for Iceberg's REST catalog. This will allow StarRocks to work with systems that implement the Iceberg REST catalog API.
Checklist:

[X] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

[FE PR Coverage Check]
:disappointed: fail : 7 / 58 (12.07%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/connector/iceberg/hive/CachedClientPool.java
0
1
00.00%
[59]

:large_blue_circle:
com/starrocks/connector/iceberg/glue/CachedGlueClientPool.java
0
1
00.00%
[56]

:large_blue_circle:
com/starrocks/connector/iceberg/CatalogLoader.java
0
13
00.00%
[45, 150, 151, 152, 153, 154, 155, 159, 160, 165, 166, 167, 168]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergUtil.java
0
8
00.00%
[70, 107, 280, 281, 282, 283, 284, 285]

:large_blue_circle:
com/starrocks/connector/iceberg/glue/GlueClientPool.java
0
1
00.00%
[27]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergMetadata.java
1
7
14.29%
[58, 59, 60, 101, 115, 116]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergRESTCatalog.java
3
20
15.00%
[29, 30, 31, 32, 33, 35, 45, 51, 56, 57, 62, 63, 65, 70, 71, 72, 73]

:large_blue_circle:
com/starrocks/catalog/IcebergTable.java
1
5
20.00%
[95, 96, 200, 201]

:large_blue_circle:
com/starrocks/connector/iceberg/hive/HiveClientPool.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergCatalogType.java
1
1
100.00%
[]

Thanks for your contribution. I want to know how you test the rest catalog based on StarRocks.

[FE PR Coverage Check]
:disappointed: fail : 10 / 58 (17.24%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/connector/iceberg/hive/CachedClientPool.java
0
1
00.00%
[59]

:large_blue_circle:
com/starrocks/connector/iceberg/glue/CachedGlueClientPool.java
0
1
00.00%
[56]

:large_blue_circle:
com/starrocks/connector/iceberg/CatalogLoader.java
0
13
00.00%
[45, 150, 151, 152, 153, 154, 155, 159, 160, 165, 166, 167, 168]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergUtil.java
0
8
00.00%
[69, 106, 278, 279, 280, 281, 282, 283]

:large_blue_circle:
com/starrocks/connector/iceberg/glue/GlueClientPool.java
0
1
00.00%
[27]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergMetadata.java
1
7
14.29%
[58, 59, 60, 101, 115, 116]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergRESTCatalog.java
3
19
15.79%
[29, 30, 33, 34, 35, 46, 52, 57, 58, 63, 64, 66, 71, 72, 73, 74]

:large_blue_circle:
com/starrocks/catalog/IcebergTable.java
4
6
66.67%
[95, 96]

:large_blue_circle:
com/starrocks/connector/iceberg/hive/HiveClientPool.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/connector/iceberg/IcebergCatalogType.java
1
1
100.00%
[]

Thanks for your contribution. I want to know how you test the rest catalog based on StarRocks.

We have a Docker image that will start up a REST catalog listening on port 8888 backed by a JDBC catalog if you want to give it a try: https://hub.docker.com/r/tabulario/iceberg-rest-jdbc/tags

run starrocks_admit_test

Thanks for your contribution. I want to know how you test the rest catalog based on StarRocks.

We have a Docker image that will start up a REST catalog listening on port 8181 backed by a JDBC catalog if you want to give it a try: https://hub.docker.com/repository/docker/tabulario/iceberg-rest. Run docker run -p 8181:8181 tabulario/iceberg-rest:latest, then set the uri catalog property to http://<IP_ADDRESS>:8181

The latest tag is arm64 that our dev machine don't support. I use the 0.1.0 amd64 to test and found that when I execute show databases from iceberg_catalog in sr , it will return empty. Is this as expected?. I have executed follow command string:
create external catalog iceberg_catalog properties ("type"="iceberg","iceberg.catalog.type"="rest","uri"="http://<IP_ADDRESS>:8181");
show databases from iceberg_catalog

Thanks for pointing out the issue with the image tag, I fixed the latest tag so it is multi-architecture now. The default config will be in-memory, so it will be empty on startup and will be cleared on restart. You can also use another catalog as a backend, there is an example of using Glue in the docs.

I have tested the rest catalog by spark writing and StarRocks reading. Thanks~

run starrocks_admit_test


not sure how to add callbacks as options

in the doc it says Callbacks can be used alongside other Options, but I cannot see how

Here's an example:
Fresco.show([
    { url: '/images/default/index/art/frans_hals.jpg', caption: "Frans Hals, Banquet of the Officers of the St George Civic Guard, 1627" },
    { url: '/images/default/index/art/rembrandt.jpg', caption: "Rembrandt, The Night Watch, 1642" },
    { url: '/images/default/index/art/charles_le_brun.jpg', caption: "Charles Le Brun, The Repentant Magdalen, 1655" },
    { url: '/images/default/index/art/reynolds.jpg', caption: "Sir Joshua Reynolds, The Countess of Dartmouth, 1757" }
], {
    afterPosition: function(position) {
        console.log("You've reached position " + position);
    }
});

I meant how to call it if I'm not setting up in javascript


Wagtail 5.2

Includes the following PR:
Wagtail 5.1 PR
Wagtail 5.2 release notes
Changes:

Adoption of classname convention for MenuItem related classes and hooks

Superseded by: https://github.com/torchbox/wagtailguide/pull/29


onPlaybackStateChanged：getPosition exception

I try to get current position in onPlaybackStateChanged, i found that onPlaybackStateChanged method is not executed in real time, but is executed every two seconds，causing the current playback progress interval I get to be 2 seconds.
How to get current position in real time ？？？

the log as below: checking the remainingSeconds value.
method: timestampToMSS; number: 63---->totalSeconds= 2, minutes= 0, remainingSeconds= 2
method: timestampToMSS; number: 63---->totalSeconds= 4, minutes= 0, remainingSeconds= 4
method: timestampToMSS; number: 63---->totalSeconds= 6, minutes= 0, remainingSeconds= 6
method: timestampToMSS; number: 63---->totalSeconds= 8, minutes= 0, remainingSeconds= 8

Working as intended. This is only triggered when the playback service reports a state change.


[T3A3][T15-A1] Zhou Fanyi

Ready for review

@fanyiii some comments added. please take a look.
please close the PR once you have noted the suggestions.


TS: code actions not properly localized

Testing: https://github.com/Microsoft/vscode/issues/37182
It looks like part of the message is still english:

This issue was moved to Microsoft/TypeScript#19611


Update README.md for hass.io setup

Without a base_topic, the z2m UI would not boot. No error displayed in logs. After adding topic, the UI boots. I am guessing the topic is actually required to properly connect to the mqtt broker.

@Koenkk It looks like I was following config instructions from before the switch to the new config system (which stores most config in the YAML file on disk). After I purged all config from the addon config UI and using the YAML file, I see the that mqtt block was overridden with auto-generated settings including the setting I mentioned in the README.
Is the target experience now that you create the config file on disk, point at it from the addon UI, and then start up? If so, i can document that.


"More tasks executed then were scheduled" error caused by zone.js

Ionic version:  (check one with "x")
[ ] 1.x
[x] 2.x
I'm submitting a ...  (check one with "x")
[x] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior:

I use xhrhttprequest send data  in ionic2 but encounter the issue: "More tasks executed then were scheduled".
This issue has been fixed in the newest zone.js and discussed in the below link.
https://github.com/angular/zone.js/pull/529
I use npm update zone.js to the version 0.7.7, but the issue is still existed.
How can i fixed or workaround this issue in ionic2?

Does ionic have the planning to update angular2 and zone?

I found the way. This issue can be fixed!
I will wait for your to release the next big updated version.


Misaligned Dock Icon on Mac [suggestion]

SQL Operations Studio Version: 0.27.3

Steps to Reproduce:

Install SQL Operations Studio on Mac
Place SQL Operations Studio on Mac Dock

A user interpreted the Mac Dock icon as misaligned. Not high priority, but may want to explore if we can tweak Dock icon in case other users also feel that way.
https://twitter.com/Philoushka/status/978677560425852929

I agree with the customer. I think just making it smaller so the bottom lines up with other apps will work.  If you look at VSCode, it's almost exactly within the top/bottom boundaries set by system preferences (see below). If we can get a similar bounds we'll look more like a native macOS app.

I believe the customer raised similar feedback in this issue https://github.com/Microsoft/sqlopsstudio/issues/857.  Though that issue is more a suggestion to completely replace the icon.

Thanks for the pull request, I'd still like to see this icon not stand out from others on my dock because it hangs so low. It looks like it goes out of bounds (bottom) compared to all others. For reference here is my dock. Where's waldo?

I know its a small thing to worry about, but there is something uniquely wrong with just the ADS icon compared to all others. Other icons seem to have a little bit of bottom padding whereas ADS maybe has 1 or 0 pixels of padding. 🙏

Closing as this refers to older icon.  Please let us know if this problem persists with the current icon.  Thanks!


Error in excution in DrQA file

Hi,
I need a clarification regarding an error found in DrQA file.
Its in the function create_word_embedding the variable vocab not found.
Will you please suggest a correction
Thanks in advance

That should be word_vocab that you get as an output from build_word_vocab above. I'll update it.
Thanks for pointing this out.

Thank you for your reply.
then after changing to word_vocab - and after executing, I am wondering I get following results with EM = 0

Hi,
Please write your personal email address, I like to extend this work.
Thank you and stay happy,
Basharat

Please suggest or if you need more information from me, please write. I haven't changed the code but downloaded three files and used: 1. glove.840B.300d.txt (5GB), 2. dataset-Squad/train-v1.0.json (28MB) 3. dataset-Squad/dev-v1.0.json (4MB).

The notebooks have been tested for SQUAD v1.1. I'll push the JSON files for convenience. I tested the DrQA notebook again and I am getting good results.

Thank you very much for constant your support - I am able to get the expected results for DrQA model.

Closing this now. Feel free to open it again if needed.


feat(ibm-schema-keywords): add new validation rule

PR summary
This commit introduces the new 'ibm-schema-keywords' validation rule which will verify that each schema and schema property are defined with only "allowed" keywords (fields), whwere the set of allowable keywords are defined in a configurable allow list.
PR Checklist
General checklist
Please make sure that your PR fulfills the following requirements:

[x] The commit message follows the Angular Commit Message Guidelines.
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)
[ ] Dependencies have been updated as needed
[ ] .secrets.baseline updated as needed?

Checklist for adding a new validation rule:

[x] Added new validation rule definition (packages/ruleset/src/rules/*.js, index.js)
[x] If necessary, added new validation rule implementation (packages/ruleset/src/functions/*.js, updated index.js)
[x] Added new rule to default configuration (packages/ruleset/src/ibm-oas.js)
[x] Added tests for new rule (packages/ruleset/test/*.test.js)
[x] Added docs for new rule (docs/ibm-cloud-rules.md)

:tada: This PR is included in version 1.12.0 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.12.0 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.3.1 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket: