Data persistence model definition

Description
As part of the new data persistence model to be implemented across Wazuh, we need to define the data model that is going to be used and shared by the Wazuh Central Components. While the Wazuh Indexer team is in charge of the creation and maintenance of the index templates, the indices and their fields, we believe their semantics depend on the context where they are being generated, hence the significance of these fields needs to be provided by the source of the data.
This new data model needs to be well documented, written and visually, by the use of diagrams, to make its understanding and maintenance as easy as possible. The data model needs to ensure that the data can be filtered using consistent criteria, such as the groups of agent they belong to.
As a result, we'll generate not only the documentation of this new data model, but the index templates that effectively represent the data model in the Indexer.
Functional requirements

The data model must cover all the indices planned for 5.0 (stateful and stateless. See parent issue).
The data model must allow Role-Based access control consistently. The data can be filtered, at least, by the groups of agents it belongs to.

Non-functional requirements

The data model must be represented in an E/R diagram.
The fields of the data model need to be well documented, indicating their meaning in the context of the source where they are generated, and providing any relevant information about their lifecycle, for example, under which conditions the data is generated or changes.
The data model must outline the visibility of the indices.

Plan

[ ] #345
[ ] #270
[x] Automate the removal of multi-fields on the generated templates
[ ] Fields semantics (provided by the sources: agent, vulnerability detector, ...)

Moved to the Feature Complete stage.


Translate into Korean

not full translate

I've just merged a Korean translation from another member. I do not speak Korean so I do not know how well the translation is. If you wouldn't mind looking through the translation and comparing to your translation that would be greatly appreciated. It seems that only some files are conflicting so if you'd like to create a PR with additional translations, that would also help!
Thanks!

Resolved by PR #694 and #689, please let me know if anything was missed.


json pagination problem

I have found a bug.
I have content_group called "participants"
I have 21 pages in "participants"
in general setting i have option "Quantity of items to be listed on frontend" set up to 10
when i try to get ...(domain).../participants.json
I got error
problem was on pagination settings
the url for next_page was impossible to create
I have omitted a problem changing Quantity of items to be listed on frontend to 100
Rendered<EMAIL_ADDRESS>(26.1ms)
Rendered<EMAIL_ADDRESS>(66.8ms)
Completed 500 Internal Server Error in 270ms (ActiveRecord: 15.1ms)
ActionView::Template::Error (Attempting to generate a URL from non-sanitized request parameters! An attacker can inject malicious data into the generated URL, such as changing the host. Whitelist and sanitize passed parameters to be secure.):
2: json.per_page posts.per_page
3: json.total_entries posts.total_entries
4: json.previous_page url_for(params.merge(page: posts.previous_page)) if posts.previous_page.present?
5: json.next_page url_for(params.merge(page: posts.next_page)) if posts.next_page.present?
6: json.entries posts.decorate do |post|
7:   json.partial! partial: 'partials/cama_post_entry', locals:{ post: post }
8: end
actionpack (5.0.1) lib/action_dispatch/routing/url_for.rb:176:in url_for' actionview (5.0.1) lib/action_view/routing_url_for.rb:96:in url_for'
camaleon_cms (2.4.2) app/views/camaleon_cms/default_theme/partials/_cama_posts_entries.json.jbuilder:5:in __usr_local_rvm_gems_ruby_______camaleon_gems_camaleon_cms_______app_views_camaleon_cms_default_theme_partials__cama_posts_entries_json_jbuilder___3652968212135170244_69974380473240' actionview (5.0.1) lib/action_view/template.rb:159:in block in render'
activesupport (5.0.1) lib/active_support/notifications.rb:166:in instrument' actionview (5.0.1) lib/action_view/template.rb:354:in instrument'
actionview (5.0.1) lib/action_view/template.rb:157:in render' actionview (5.0.1) lib/action_view/renderer/partial_renderer.rb:343:in render_partial'
actionview (5.0.1) lib/action_view/renderer/partial_renderer.rb:311:in block in render' actionview (5.0.1) lib/action_view/renderer/abstract_renderer.rb:42:in block in instrument'
activesupport (5.0.1) lib/active_support/notifications.rb:164:in block in instrument' activesupport (5.0.1) lib/active_support/notifications/instrumenter.rb:21:in instrument'
activesupport (5.0.1) lib/active_support/notifications.rb:164:in instrument' actionview (5.0.1) lib/action_view/renderer/abstract_renderer.rb:41:in instrument'
actionview (5.0.1) lib/action_view/renderer/partial_renderer.rb:310:in render' actionview (5.0.1) lib/action_view/renderer/renderer.rb:47:in render_partial'
actionview (5.0.1) lib/action_view/renderer/renderer.rb:21:in render' actionview (5.0.1) lib/action_view/helpers/rendering_helper.rb:32:in render'
jbuilder (2.6.1) lib/jbuilder/jbuilder_template.rb:127:in _render_partial' jbuilder (2.6.1) lib/jbuilder/jbuilder_template.rb:121:in _render_partial_with_options'
jbuilder (2.6.1) lib/jbuilder/jbuilder_template.rb:214:in _render_explicit_partial' jbuilder (2.6.1) lib/jbuilder/jbuilder_template.rb:21:in partial!'
camaleon_cms (2.4.2) app/views/camaleon_cms/default_theme/post_type.json.jbuilder:4:in block in __usr_local_rvm_gems_ruby_______camaleon_gems_camaleon_cms_______app_views_camaleon_cms_default_theme_post_type_json_jbuilder__1596179320828063274_69974381390920' jbuilder (2.6.1) lib/jbuilder.rb:38:in block in set!'
jbuilder (2.6.1) lib/jbuilder.rb:266:in block in _merge_block' jbuilder (2.6.1) lib/jbuilder.rb:305:in _scope'
jbuilder (2.6.1) lib/jbuilder.rb:266:in _merge_block' jbuilder (2.6.1) lib/jbuilder.rb:38:in set!'
jbuilder (2.6.1) lib/jbuilder/jbuilder_template.rb:99:in set!' jbuilder (2.6.1) lib/jbuilder.rb:66:in method_missing'
camaleon_cms (2.4.2) app/views/camaleon_cms/default_theme/post_type.json.jbuilder:3:in __usr_local_rvm_gems_ruby_______camaleon_gems_camaleon_cms_______app_views_camaleon_cms_default_theme_post_type_json_jbuilder__1596179320828063274_69974381390920' actionview (5.0.1) lib/action_view/template.rb:159:in block in render'
activesupport (5.0.1) lib/active_support/notifications.rb:166:in instrument' actionview (5.0.1) lib/action_view/template.rb:354:in instrument'
actionview (5.0.1) lib/action_view/template.rb:157:in render' actionview (5.0.1) lib/action_view/renderer/template_renderer.rb:54:in block (2 levels) in render_template'
actionview (5.0.1) lib/action_view/renderer/abstract_renderer.rb:42:in block in instrument' activesupport (5.0.1) lib/active_support/notifications.rb:164:in block in instrument'
activesupport (5.0.1) lib/active_support/notifications/instrumenter.rb:21:in instrument' activesupport (5.0.1) lib/active_support/notifications.rb:164:in instrument'
actionview (5.0.1) lib/action_view/renderer/abstract_renderer.rb:41:in instrument' actionview (5.0.1) lib/action_view/renderer/template_renderer.rb:53:in block in render_template'
actionview (5.0.1) lib/action_view/renderer/template_renderer.rb:61:in render_with_layout' actionview (5.0.1) lib/action_view/renderer/template_renderer.rb:52:in render_template'
actionview (5.0.1) lib/action_view/renderer/template_renderer.rb:14:in render' actionview (5.0.1) lib/action_view/renderer/renderer.rb:42:in render_template'
actionview (5.0.1) lib/action_view/renderer/renderer.rb:23:in render' actionview (5.0.1) lib/action_view/rendering.rb:103:in _render_template'
actionpack (5.0.1) lib/action_controller/metal/streaming.rb:217:in _render_template' actionview (5.0.1) lib/action_view/rendering.rb:83:in render_to_body'
actionpack (5.0.1) lib/action_controller/metal/rendering.rb:52:in render_to_body' actionpack (5.0.1) lib/action_controller/metal/renderers.rb:142:in render_to_body'
actionpack (5.0.1) lib/abstract_controller/rendering.rb:26:in render' actionpack (5.0.1) lib/action_controller/metal/rendering.rb:36:in render'
actionpack (5.0.1) lib/action_controller/metal/instrumentation.rb:44:in block (2 levels) in render' activesupport (5.0.1) lib/active_support/core_ext/benchmark.rb:12:in block in ms'
/usr/local/rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/benchmark.rb:308:in realtime' activesupport (5.0.1) lib/active_support/core_ext/benchmark.rb:12:in ms'
actionpack (5.0.1) lib/action_controller/metal/instrumentation.rb:44:in block in render' actionpack (5.0.1) lib/action_controller/metal/instrumentation.rb:87:in cleanup_view_runtime'
activerecord (5.0.1) lib/active_record/railties/controller_runtime.rb:25:in cleanup_view_runtime' actionpack (5.0.1) lib/action_controller/metal/instrumentation.rb:43:in render'
meta-tags (2.4.0) lib/meta_tags/controller_helper.rb:20:in render' camaleon_cms (2.4.2) app/controllers/camaleon_cms/frontend_controller.rb:57:in post_type'
actionpack (5.0.1) lib/action_controller/metal/basic_implicit_render.rb:4:in send_action' actionpack (5.0.1) lib/abstract_controller/base.rb:188:in process_action'
actionpack (5.0.1) lib/action_controller/metal/rendering.rb:30:in process_action' actionpack (5.0.1) lib/abstract_controller/callbacks.rb:20:in block in process_action'
activesupport (5.0.1) lib/active_support/callbacks.rb:126:in call' activesupport (5.0.1) lib/active_support/callbacks.rb:506:in block (2 levels) in compile'
activesupport (5.0.1) lib/active_support/callbacks.rb:455:in call' activesupport (5.0.1) lib/active_support/callbacks.rb:101:in run_callbacks'
activesupport (5.0.1) lib/active_support/callbacks.rb:750:in _run_process_action_callbacks' activesupport (5.0.1) lib/active_support/callbacks.rb:90:in run_callbacks'
actionpack (5.0.1) lib/abstract_controller/callbacks.rb:19:in process_action' actionpack (5.0.1) lib/action_controller/metal/rescue.rb:20:in process_action'
actionpack (5.0.1) lib/action_controller/metal/instrumentation.rb:32:in block in process_action' activesupport (5.0.1) lib/active_support/notifications.rb:164:in block in instrument'
activesupport (5.0.1) lib/active_support/notifications/instrumenter.rb:21:in instrument' activesupport (5.0.1) lib/active_support/notifications.rb:164:in instrument'
actionpack (5.0.1) lib/action_controller/metal/instrumentation.rb:30:in process_action' actionpack (5.0.1) lib/action_controller/metal/params_wrapper.rb:248:in process_action'
activerecord (5.0.1) lib/active_record/railties/controller_runtime.rb:18:in process_action' actionpack (5.0.1) lib/abstract_controller/base.rb:126:in process'
actionview (5.0.1) lib/action_view/rendering.rb:30:in process' actionpack (5.0.1) lib/action_controller/metal.rb:190:in dispatch'
actionpack (5.0.1) lib/action_controller/metal.rb:262:in dispatch' actionpack (5.0.1) lib/action_dispatch/routing/route_set.rb:50:in dispatch'
actionpack (5.0.1) lib/action_dispatch/routing/route_set.rb:32:in serve' actionpack (5.0.1) lib/action_dispatch/journey/router.rb:39:in block in serve'
actionpack (5.0.1) lib/action_dispatch/journey/router.rb:26:in each' actionpack (5.0.1) lib/action_dispatch/journey/router.rb:26:in serve'
actionpack (5.0.1) lib/action_dispatch/routing/route_set.rb:725:in call' actionpack (5.0.1) lib/action_dispatch/middleware/static.rb:136:in call'
rack (2.0.1) lib/rack/etag.rb:25:in call' rack (2.0.1) lib/rack/conditional_get.rb:25:in call'
rack (2.0.1) lib/rack/head.rb:12:in call' rack (2.0.1) lib/rack/session/abstract/id.rb:222:in context'
rack (2.0.1) lib/rack/session/abstract/id.rb:216:in call' actionpack (5.0.1) lib/action_dispatch/middleware/cookies.rb:613:in call'
activerecord (5.0.1) lib/active_record/migration.rb:553:in call' actionpack (5.0.1) lib/action_dispatch/middleware/callbacks.rb:38:in block in call'
activesupport (5.0.1) lib/active_support/callbacks.rb:97:in __run_callbacks__' activesupport (5.0.1) lib/active_support/callbacks.rb:750:in _run_call_callbacks'
activesupport (5.0.1) lib/active_support/callbacks.rb:90:in run_callbacks' actionpack (5.0.1) lib/action_dispatch/middleware/callbacks.rb:36:in call'
actionpack (5.0.1) lib/action_dispatch/middleware/executor.rb:12:in call' actionpack (5.0.1) lib/action_dispatch/middleware/remote_ip.rb:79:in call'
actionpack (5.0.1) lib/action_dispatch/middleware/debug_exceptions.rb:49:in call' web-console (3.4.0) lib/web_console/middleware.rb:135:in call_app'
web-console (3.4.0) lib/web_console/middleware.rb:20:in block in call' web-console (3.4.0) lib/web_console/middleware.rb:18:in catch'
web-console (3.4.0) lib/web_console/middleware.rb:18:in call' actionpack (5.0.1) lib/action_dispatch/middleware/show_exceptions.rb:31:in call'
railties (5.0.1) lib/rails/rack/logger.rb:36:in call_app' railties (5.0.1) lib/rails/rack/logger.rb:24:in block in call'
activesupport (5.0.1) lib/active_support/tagged_logging.rb:69:in block in tagged' activesupport (5.0.1) lib/active_support/tagged_logging.rb:26:in tagged'
activesupport (5.0.1) lib/active_support/tagged_logging.rb:69:in tagged' railties (5.0.1) lib/rails/rack/logger.rb:24:in call'
sprockets-rails (3.2.0) lib/sprockets/rails/quiet_assets.rb:13:in call' request_store (1.3.1) lib/request_store/middleware.rb:9:in call'
actionpack (5.0.1) lib/action_dispatch/middleware/request_id.rb:24:in call' rack (2.0.1) lib/rack/method_override.rb:22:in call'
rack (2.0.1) lib/rack/runtime.rb:22:in call' activesupport (5.0.1) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in call'
actionpack (5.0.1) lib/action_dispatch/middleware/executor.rb:12:in call' actionpack (5.0.1) lib/action_dispatch/middleware/static.rb:136:in call'
rack (2.0.1) lib/rack/sendfile.rb:111:in call' railties (5.0.1) lib/rails/engine.rb:522:in call'

Thanks for the report, I will fix until Friday please.

Solved!
thanks for reporting.
Released new version (<IP_ADDRESS>) with the fix.


Compiling in Ubuntu 14.04 throws error

Hi,
Thanks for the great tool. It's an ideal tool for my use case requiring a direct replacement for redis hash with high scalability. May I check with you if the installation of this in Ubuntu 14.04 is just compiling using 'storage_engine=leveldb make'. If so, I get this error when I run a make:
g++ -c -Wall -g -O2 -fPIC -D__STDC_FORMAT_MACROS -DARDB_VERSION='"0.8.2"'  -I./ -I./common -I/home/user/projects/ds/ardb/src/../deps/cpp-btree -I/home/user/projects/ds/ardb/src/../deps/lua/src -I/home/user/projects/ds/ardb/src/../deps/snappy-1.1.2  -I/home/user/projects/ds/ardb/src/../deps/leveldb-1.16.0/include engine/leveldb_engine.cpp -o engine/leveldb_engine.o
engine/leveldb_engine.cpp: In member function ‘virtual void ardb::LevelDBLogger::Logv(const char*, __va_list_tag*)’:
engine/leveldb_engine.cpp:48:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (len < sizeof(logbuf))
^
echo ">>>>> Building JEMALLOC" &&
cd /home/user/projects/ds/ardb/src/../deps &&
tar jxf /home/user/projects/ds/ardb/src/../deps/jemalloc-3.6.0.tar.bz2 &&
cd /home/user/projects/ds/ardb/src/../deps/jemalloc-3.6.0 &&
patch < ../../jemalloc.patch &&
autoconf configure.ac > configure 2> autoconf.out &&
./configure    > configure.out &&
make build_lib_static &&
echo "<<<<< Done building JEMALLOC"

Building JEMALLOC
patching file configure.ac
make[1]: *** [/home/user/projects/ds/ardb/src/../deps/jemalloc-3.6.0/lib/libjemalloc.a] Error 127
make[1]: Leaving directory `/home/user/projects/ds/ardb/src'
make: *** [all] Error 2

EDIT: If this might help
OS: Ubuntu 14.04 LTS
CPU: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Kernel: Linux 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
gcc: gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
Am running the build as a non-root user.
If I'm doing something wrong here, please help me with the correct way to build, install and run in ubuntu 14.04.

Saved me some time too, thank you :)

Saved me some time too ,and too much warn output 谢谢


Update some lua code, then black screen

axmol version: latest verison of 'dev' branch
devices test on: MacBookPro M3 Max
developing environments

Xcode version: 15.3
cmake version: 3.29.2
Steps to Reproduce:

git pull the latest dev branch code of axmol
Execute ./setup.ps1 under axmol
Execute axmol new -p org.axmol.hellolua -d ~\workspace\games\ -l lua HelloLua to create a new project
Execute axmol build -p osx -a arm64 -c to build the HelloLua project
Execute axmol run, it will build and show me the axmol hello world scene.
Duplicate the MainScene.lua, name the new file MainSceneX.lua, update the code in MainSceneX.lua with the new class name "MainSceneX"
Change the line 8 of AppBase.lua in Content/src/packages/mvc/, make it 'defaultSceneName = "MainSceneX"
Execute axmol run again, then I got black screen
Not only with this reproduce process, whenever I add some new lua files, tried to import them to the MainScene, I got the black screen, I can only write the code into MainScene.lua, that is the only way it take effect.

While, on macos, the develop workflow should stupid, will build resources to HelloCpp.app, you need always regenerate cmake when you add new resources, due to it's no chance to use symlink like windows and linux.

―
Reply to this email directly, view it on GitHubhttps://github.com/axmolengine/axmol/issues/1856#issuecomment-2081293262, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABVHOJ3NM2A2LU27GJ7NHTDY7RLOLAVCNFSM6AAAAABG34ASEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGI4TGMRWGI.
You are receiving this because you commented.Message ID: @.***>

@jie-meng Tested this, and cannot reproduce the issue.
Please show the contents of the MainSceneX.lua file.

I just pushed the project code to: https://github.com/jie-meng/HelloLua.
And this is MainSceneX.lua: https://github.com/jie-meng/HelloLua/blob/main/Content/src/app/views/MainSceneX.lua

I just pushed the project code to: https://github.com/jie-meng/HelloLua. And this is MainSceneX.lua: https://github.com/jie-meng/HelloLua/blob/main/Content/src/app/views/MainSceneX.lua

I cloned that repository, and built it for MacOS arm64, and it ran correctly.  The command used to generate the project is: axmol build -p osx -a arm64 -c -f, then opened the project up in Xcode and clicked the build/run command.
The only differences I can see is that you're using Darwin 23.4.0, and Xcode 15.3, where as I'm using Darwin 22.6.0 and Xcode 15.0.1.

It works when I type axmol build -p osx -a arm64 -c -f.

It works when I type axmol build -p osx -a arm64 -c -f.

That means the project was never updated after MainSceneX.lua was created.  The -f is to force cmake to regenerate the xcode project, which is always required when new files are added.

This document
https://github.com/axmolengine/axmol/blob/dev/docs/DevSetup.md
said:
for macOS arm64(M1) axmol build -p osx -a arm64 -c

That is only for initially generating the project, but cmake does not detect changes in the Contents folder, which is where the LUA files are, so the only way to update the project is to force it with -f.  This should be added to the documentation.

So what's the meaning of "-c" & "-f"?

-c just generates the project files, and does not build the project
-f forces cmake to update the project files even if cmake does not detect changes

@jie-meng Please close this issue if it is now resolved for you.

On MacOS, there is another problem... Cannot print log.
I tried this:
local TestScene = class("TestScene", cc.load("mvc").ViewBase)

function TestScene:onCreate()
    -- add background image
    display.newSprite("HelloWorld.png")
        :move(display.center)
        :addTo(self)

    -- add HelloWorld label
    cc.Label:createWithSystemFont("Test", "Arial", 40)
        :move(display.cx, display.cy + 200)
        :addTo(self)

    printLog("TestScene", "TestLog")
end

return TestScene

The "printLog" in functions.lua uses standard lua function "print", I don't know if it has bridged to MacOS app standard terminal output or not.
Nothing printed.


[21.11] consul: 1.10.3 -> 1.10.10

Description of changes
CVE fixes: Fixes #166623. Fixes #172481.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

[x] NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

PR #172618

@GrahamcOfBorg build consul consul.tests

Result of nixpkgs-review pr 172618 run on x86_64-darwin 1

1 package built:

consul

Tests pass on all but aarch64-darwin where I'm not sure if it's supposed to build; build error is
https://logs.nix.ci/?key=nixos/nixpkgs.172618&attempt_id=a9f2750c-f22e-4857-aef0-4389ca51f95a
vendor/github.com/shirou/gopsutil/v3/cpu/cpu_darwin_cgo.go:13:5: error: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Werror,-Wundef-prefix=TARGET_OS_]
#if TARGET_OS_MAC
    ^
1 error generated.

Merging for security update.


Issues with new changes DevOps Pipeline

The changes are not backward compatible. Our pipelines started failing as soon as you are making chnages. It failed on June 14 as well. that worked after the changes were reverted. It will be good, if you allow the clients to control the version so that they can work out by keeping old version while you guys try to test out the new changes. As of now, it breaks our PROD systems.
error: 1
failed: Reading the import configuration. Please wait.
Failed to Load the Import Configuration : Config File Missing
Failed to Load the Import Configuration : Config File Missing
Microsoft PowerPlatform CLI
Version: 1.15.9+gf60807f
Error: Failed to Load the Import Configuration : Config File Missing
Usage: pac package deploy [--logFile] [--logConsole] --package
--logFile                   Log file path (alias: -lf)
--logConsole                Output log to console (alias: -c)
--package                   path to a package dll or zip file with a package (alias: -p)
1 more errors. Click on expand view in the context menu to view complete logs.

@amitvirendra what are the more detailed errors, both in the log, but also in the DeployPackage detail log that is saved to the AzDevOps run's artifact store?
RE: versioning: It was a conscious choice by the team, as it has the least amount of pain on existing customers. Since both the old PS based and the pac CLI based versions do depend on having an initial ToolInstaller, there is no mix and match between e.g. @0 ToolInstaller and a @1 DeployPackage task.

We are having the exact same issue ! this is stoping us to do deployment , we need this fixed urgently

@amitvirendra @Faranakt we understand the urgency. Here's what I've gleaned so far from the above logs:

any error in the PackageDeployer code itself (this is C# code shared between the former PS PP-BT and the C# pac CLI now) is expected to be written to the detailed deployer log, e.g. indicated in the log above as:
--logFile C:\_a\out\DeployPackage\deploy-package\DeploymentPackage-2022-06-30T19-34-33.281Z.log
This log will assist package/pipeline owners to further debug.
to add insult to injury, even though the new PP-BT now by default ensures that this detail log is always a) captured and b) uploaded into AzDevOps run logs (as artifact for build pipelines, or as separate log to the overall log store for release pipelines), that code path ended up erroring out, see the nodejs runtime errors.
the one odd thing in the log is the double mentioning of "Failed to Load the Import Configuration : Config File Missing":
while not useful to the user, it helped me correlate with the PD core src code that this happens only when the package dll could not be loaded via reflection (std procedure for PD). This hints that the newer version of the PD core lib that ships with this PP-BT wasn't able to load the DLL, typ, that is an issue with missing dependent assemblies

We have 2 tracking bugs on our end:

highest prio: fix the missing logs issue, so users can diagnose e.g. why the pkg dll could not be loaded
AB#2776373
fix PD to not double print the same error, but actually indicate more why the dll failed to load
AB#2776344

The first fix should be ready later today PDT

@amitvirendra @Faranakt the fix to at be able to capture and upload the PD core library's detail log is now deployed as 1.0,82.
This log file should help to further investigate WHY the package dll failed to load. Either quote parts here, or forward to my MSFT corp email address with my alias, davidjen

In build pipelines, the log is added into a separate "DeployPackage" artifact, available from the job's summary, see: artifacts
In release pipelines, the log is part of the "Download all logs" zip file, named as: <packageName>-<timestamp>.log

It will be good, if you allow the clients to control the version so that they can work out by keeping old version while you guys try to test out the new changes. As of now, it breaks our PROD systems.

I agree, devops even allow for versioning of tasks. Sadly, this has not been used.

here are all logs. I masked the sensitive info
ReleasePipeline_Logs_07052022.zip
.

@amitvirendra thank you for the log file, this confirms the issue.
We have an internal tracking bug:
AB#2776373: PP.BT: PackageDeploy: PD-Core fails to load the pkg dll - Could not load file or assembly 'Microsoft.Xrm.Tooling.PackageDeployment.CrmPackageExtentionBase, Version=<IP_ADDRESS>
I'll leave this issue open until we have a fix ready. I will resolve duplicate issues to this one.

Part of PP-BT v2.0.3 and newer
https://aka.ms/pp-bt-migrate-to-v2


JAV-153 - Separate samples provider and consumer side code.

Coverage increased (+0.04%) to 84.691% when pulling 37fcf184b75042fd45bfeed811eebee34cb48f92 on TankTian:master into b3edf3825d3c66cc5c5dd71555a4fae23f3309aa on ServiceComb:master.


doc(managing): Remove managing disk usage page

afaik (and someone correct me, please, if I have this wrong), the kubelet handles "garbage collection" for dead containers and stale/unused images.  In contrast to Deis v1.x, I don't think there's anything to really say on the topic of managing disk usage.

Related question - does it make sense to add docs that explain how to apply restrictions on the disk resource for the minio pod? Although it uses ephemeral disk, it could get large.

@arschles I'm wondering if that's an advanced enough topic that it belongs in that component's own docs?  idk

@krancour good idea. I've created https://github.com/deis/minio/issues/88 to track


fix: bug 1639

formatMessageId updates

:tada: This PR is included in version 0.323.7 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add Visual Studio 2022 projects

For now the 2019 projects can be upgraded in place.

For now, you can just upgrade them in place.


identity 979

Better to sort the claims in the backend components. E.g. ApplicationManagementServiceImpl.getAllLocalClaimUris()


Unable to connect to Azure Services from VSCode.

I'm working in Company provided Mac. And found out that I'm unable to connect to Azure services from VS Code provided Azure extension. Whereas I was able to signin few days back but from today despite several attempts unable to login to Azure Services

OS Version : Mac OS Ventura 13.2.1
Version: 1.76.0
Commit: 92da9481c0904c6adfe372c12da3b7748d74bdcb
Date: 2023-03-01T10:23:06.752Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 22.3.0
Sandboxed: No

/extAzure


Give more nodes to SOC8 ha jobs

For HA that is 2 for pacemaker cluster, 3 for ceph and 1 as compute node.
ceph nodes cannot be used for anything else as they have SP2 system.
compute node cannot be in the cluster.

For HA that is 2 for pacemaker cluster, 3 for ceph and 1 as compute node.

Thats true, but only if the run is configured to deploy ceph.
By increasing the nodenumbers you are actually changing the defaults: If the job detects enough nodes, then it will default to ceph.
Is this what you wanted to change? To deploy ceph by default?

ceph nodes cannot be used for anything else as they have SP2 system.
compute node cannot be in the cluster.

correct but unrelated to the actual change

I really think this change is affecting only "cloud-mkcloud-job-ha-template.yaml" and only for Cloud8, as the other HA jobs have explicit value for nodenumber and for other cloudsources I'm keeping the new variable at the original value

@jdsn I'm not sure if I understand your point correctly but this line: https://github.com/SUSE-Cloud/automation/blob/master/scripts/qa_crowbarsetup.sh#L3213 causes mkcloud to "prepare" for ceph deployment if there are more than 2 nodes and you didn't explicitly state want_ceph=0 and/or storage_method!=default. I think that this logic is too "optimistic" about ceph deployments as it is quite common that we have more than 2 nodes and we don't plan to deploy ceph. Before cloud8 it was not very visible as the nodes prepared for ceph could be used for other roles. The only (?) difference was that there were additional volumes allocated for ceph disks, which was also a waste of resources if we didn't actually deploy ceph on those volumes.

I really think this change is affecting only "cloud-mkcloud-job-ha-template.yaml" and only for Cloud8

Well, looking only at this PR that might be the case. But you introduce a new variable and set it to a value even if it is no used. That may seem as a bug or at least unwanted (no need for a variable in this case).

as the other HA jobs have explicit value for nodenumber

So some jobs are overriding the variable. Some not.
Why this is, is not documented by your change. From my POV this should be an inline comment to show how this is supposed to work.
Alternative (without the need to document it everywhere):

define 2 variables nodenumer: x (previous default) and nodenumber_ha: y (new ha default)
in the templates use the nodenumer_ha variable for ha jobs, and the nodenumber variable for non-ha.

This way we make a clear distinction what variable is used for what and why the templates differ.

Before cloud8 it was not very visible as the nodes prepared for ceph could be used for other roles.

@skazi0 So effectively this means that we should rather change the default behaviour of this finding the default. This is because starting of cloud 8 the restriction also changed, that ceph nodes could be used for other roles. So changing this along side would be at least consistent.
Would you agree?

@jdsn as I mentioned I think that the original logic of detecting if cloud should be prepared for ceph needs to be changed, even in cloud7 because it was too "optimistic" or "eager". I would rather simplify it so that by default we assume that ceph will NOT be deployed unless it's stated explicitly (via want_ceph or storage_method). IMO, the "smartness" of current implementation doesn't bring too much value. Of course we would need to review all of the scenarios if some are not relying on this implicit "enabling" of ceph but it should not be too much effort.

Closing in favor of https://github.com/SUSE-Cloud/automation/pull/2027


xgboost-spark complie and yarn cluster env problem

Does the newest version  of xgboost-spark  require that the env of  the machines in yarn cluster is g++4.8 or higher?
I complie xgboost-jvm in local linux machine, and running it on yarn cluster failed. Our cluster machines' gcc version is lower than 4.8, but upgrading all of the machines‘ gcc version is impossible. The spark version is 2.1.0.
I also complie low xgboost-jvm version v0.60 on cluster machine, but run failed too.

I think this is not related to the Spark version, but rather to the XGBoost itself. @khotilov can you comment?

@lxmly we don't know what error you are seeing while trying to run on a machine with older gcc, and whether those are different linux systems to your local machine. But if the systems are similar enough and GLIBC versions are compatible, try shipping some of the shared libraries from your local machine that xgboost depends on (see with ldd libxgboost.so) together with libxgboost.so.


A few more cleanup items

fix bash simple variable
comments should start with a space
removed unused functions, parameters, and code

/lgtm

/retest

/retest


How to disable local package download but use %UserProfile.nuget\packages% cache instead?

Is it possible?
And how if possible?

What do you mean? We use the cache.

I mean do not create packages folder at all but use %UserProfile.nuget\packages% in order to resolve packages

There is an option for that "storage:none" - you can set that per group.
But you should only do that for new style sdk projects
Am 09.12.2017 10:49 schrieb "Andrii Chebukin"<EMAIL_ADDRESS>
I mean do not create packages folder at all but use
%UserProfile.nuget\packages% in order to resolve packages
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fsprojects/Paket/issues/2933#issuecomment-350440945,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNC_TWCv9_xjm4zuJ5kbh3yN-tp7mks5s-lfBgaJpZM4Q7guZ
.

So that I just put this at the top of paket file?
Looks like it works

Yes if only have one group then top of deps file will do
Am 10.12.2017 10:24 schrieb "Andrii Chebukin"<EMAIL_ADDRESS>
So that I just put this at the top of paket file?
Looks like it works
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fsprojects/Paket/issues/2933#issuecomment-350535409,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNK5HLUl8HXadjHdXUhp79ygfApOGks5s-6M8gaJpZM4Q7guZ
.


Incompatible with 'Valkyrie Skies for 1.20.1'

Describe
With both Valkyrie Skies and Leawind's Third Person enabled, the player cannot place(use) blocks.
Reproduce

Install Valkyrie Skies and Leawind's Third Person on Forge 1.20.1
Join a world and place a block in the third person

Expected behaviour
The player should place the block as normal.
Additional context
On placing the block, no logs are output. No unusual logs on Valkyrie Skies. No other bugs were noticed with Valkyrie Skies.
Placing blocks in first person, vanilla third person and duration smooth transition from first person to third person works normally. On the first seconds of world join, sometimes it may be able to place one block, on the wrong camera place.
A rough guess: Valkyrie Skies implemented a special ray cast for checking to place blocks on physics entities, maybe it conflicts with Leawind's Third Person.

I confirm

Also experiencing this issue

I too have this issue. I am using a custom fork for Valkyrian skies on 1.20.1

This is the issue I experienced. It's odd that it applies even when not interacting with Valkyrien Skies builds.

Also have the same issue on forge 1.20.1


Inconsistent mapping rules status code

If you have only mapping rule  [GET] /ip, and you try to request [POST] /ip it returns 403, but if you try [GET] /incorrect it returns 404.
I think those cases should return the same status code.

@pehala do you know what apicast does in the same situation?


Bc compliance

what

BridgeCrew compliance checks fix
readme updated
code clean up
workflows updated
default behaviour changed: is_multi_region_trail enabled by default

why

To be able to position our modules as standards compliant
Providing explicit note about policy attached directly to user
removed unnecessary providers dependencies
to stay in sync with all repos
To comply BridgeCrew check

references

https://docs.bridgecrew.io/docs/logging_1

/test all

/test all


Endpoint.SerializerContext<TContext>() does not serialize as intended due to default Config.SerializerOpts

If Endpoint.SerializerContext<TContext>() is called inside Endpoint.Configure() the JsonSerializerContext created is constructed using the default new JsonSerializerOptions() from Config.SerializerOpts
The basic issue if that Endpoint.Configure() during BuildEndpointDefinitions() in the following flow:
public static IServiceCollection AddFastEndpoints(this IServiceCollection services, IEnumerable<Assembly>? endpointAssmeblies = null)

.. private static EndpointDefinition[] BuildEndpointDefinitions(IServiceCollection services, IEnumerable<Assembly> assemblies)

.... instance.Configure();

...... Config.SerializerOpts - Has not been set...

But Config.SerializerOpts isn't set until the call to MapFastEndpoints() in this flow:
public static WebApplication UseFastEndpoints(this WebApplication app, Action<Config>? configAction = null)

.. public static IEndpointRouteBuilder MapFastEndpoints(this IEndpointRouteBuilder app, Action<Config>? configAction = null)

.... Resolves IOptions<JsonOptions> and sets Config.SerializerOpts

I tried taking a stab at fixing this prior to opening this issue but it's late and I don't see a straightforward path that wouldn't involve a discussion. I'm going to submit a PR momentarily that updates the Web project so this is easily visible/reproducible in the GET api/admin/login/v1 endpoint.

good catch mate!
fixed in v3.12.1-beta1
thanks for reporting :-)

Confirmed fixed in v3.12.1-beta1


docs(webrisk-samples): add new samples (search, threat diff) and refa…

…ctor existing

Adding two new samples:
Search Hashes
Compute Threat List diff

Refactor existing samples and updated tests to match Java standard guide.

@lesv PTAL at the changes. Thanks.


HG_Get_input very slow with eager mode + embedded bulk data

Reported by Xuezhao. Should be easy to at least reproduce the problem - send an RPC with a bulk handle of size X (that fits in the eager message).

I could reproduce it by adding hg_time_get_current() counters in mercury_rpc_cb.c line 1083 around the get_input and run the hg_test_perf test (which also tests eager messages)

So here's what happens when an eager buffer is decoded:

an output buffer is allocated
hg_proc_raw is called on the full buffer
in hg_proc_raw, each byte is processed individually
for each byte, hg_proc_uint8_t is called
in hg_proc_uint8_t (hg_proc_hg_uint8_t), hg_proc_memcpy is called (unlike the others, this isn't inlined)
hg_proc_memcpy does buffer management followed by a call to hg_proc_buf_memcpy, which does some pointer checks and calls memcpy

A few observations:

I think it's reasonable to assume that the lifetime of the eager buffer is the same as the lifetime of the input pointer - until the handle is destroyed. Then, a direct pointer to the RPC buffer is provided to the proc caller and the caller can dup the buffer if they really need a copy of the data. This by itself will solve the HG_Get_input performance woes Xuezhao has.
what's the difference between hg_proc_raw and hg_proc_memcpy? Endianness of the source is preserved in each case if I'm reading the code right (copying byte-by-byte vs. memcpy), so we could just replace all hg_proc_raw's with hg_proc_memcpy instead? Since hg_proc_memcpy isn't inlined, it seems like we're paying a significant overhead when running an hg_proc_raw.
Related to #90, a proc pass to get the size of the decoded output would prevent the multiple allocations + copying that the current approach does, esp. since size checks and resulting buffer reallocations happen on every distinct thing proc'd (and every byte for eager buffers). That would help to reduce the allocation overheads we've observed.

I think I got it fixed. I have to measure set_input as well because we likely have the same problem there. For 2, I think hg_proc_raw is the problem anyway, the idea with hg_proc is that we wanted to keep xdr encoding if there is a need for it (not in this case though). For 3, I agree the latency trade-off is better if there are multiple allocations, it is worse if there is no allocation, might want to have a mix of these 2 solutions...

Pushed the fix, @liuxuezhao please let me know if that improves things

Hi,
Thanks for the quick action, I confirm the get_input is faster now.
BTW, got a compiling error on my platform of:
/usr/local/include/mercury_proc.h:311: error: multiple storage classes in declaration specifiers
Can be resolved by remove the "static"

BTW, I tested the NA_CCI_USE_POLL, the CPU usage rate does decrease, but performance down graded. (DAOS KV update ops rate decreased from about 8K ops/S to 6K).
The mercury benchmark also report larger round-trip time if turning on CCI_USE_POLL.

I fixed some more warnings in 2daeee7085c80af4370c7e8583be2d7246ac7160, should be better now. Regarding the NA_CCI_USE_POLL option, ok, we should move that discussion to another thread then.


Faster CI for contracts

What is the purpose of the change
Make it easier to compile the x86 contracts on CI
Brief Changelog

Only running the optimizer once

Testing and Verifying
CI should pass
Documentation and Release Note

Does this pull request introduce a new feature or user-facing behavior changes? (/ no)
Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? (/ no)
How is the feature or change documented? (not applicable )

can we add some comments to this, I don't really understand it tbh, I imagine next editor will have similar problems

sorry, yeah, this is not done. I'll make it a draft

My objective here is to execute the optimize and upload artifacts steps only once per workdir, but then check all the diffs.
My initial experiments are not working, so I'll move on and come back to this later

I'm gonna close this for now. Need to revisit this eventually. Added an issue to track it: https://github.com/osmosis-labs/osmosis/issues/4123

I'm gonna close this for now. Need to revisit this eventually. Added an issue to track it: https://github.com/osmosis-labs/osmosis/issues/4123


[tool_crash] FileSystemException: Cannot create link, OS Error: Incorrect function., errno = 1

Command
flutter pub get

Steps to Reproduce

flutter pub add path_provider
Code is not on C drive on windows
This happens for a ton of stuff with Flutter.  All my development is on an external USB drive.  There is something it doesn't like about mapping to external drives.

Logs
FileSystemException: Cannot create link, OS Error: Incorrect function.
, errno = 1
#0      _Link.throwIfError (dart:io/link.dart:263:7)
#1      _Link.createSync (dart:io/link.dart:184:5)
#2      ForwardingLink.createSync (package:file/src/forwarding/forwarding_link.dart:20:16)
#3      ForwardingLink.createSync (package:file/src/forwarding/forwarding_link.dart:20:16)
#4      _createPlatformPluginSymlinks (package:flutter_tools/src/flutter_plugins.dart:1029:12)
#5      createPluginSymlinks (package:flutter_tools/src/flutter_plugins.dart:964:5)
#6      refreshPluginsList (package:flutter_tools/src/flutter_plugins.dart:1055:5)
<asynchronous suspension>
#7      FlutterProject.ensureReadyForPlatformSpecificTooling (package:flutter_tools/src/project.dart:316:5)
<asynchronous suspension>
#8      PackagesGetCommand.runCommand (package:flutter_tools/src/commands/packages.dart:177:5)
<asynchronous suspension>
#9      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1161:27)
<asynchronous suspension>
#10     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#11     CommandRunner.runCommand (package:args/command_runner.dart:209:13)
<asynchronous suspension>
#12     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:281:9)
<asynchronous suspension>
#13     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#14     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:229:5)
<asynchronous suspension>
#15     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:62:9)

[✓] Flutter (Channel stable, 2.10.2, on Microsoft Windows [Version 10.0.22000.469], locale en-CA)
    • Flutter version 2.10.2 at C:\tools\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 097d3313d8 (7 weeks ago), 2022-02-18 19:33:08 -0600
    • Engine revision a83ed0e5e3
    • Dart version 2.16.1
    • DevTools version 2.9.2

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.1)
    • Android SDK at C:\Users\Rob\AppData\Local\Android\Sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.

[✗] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[!] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.9)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.11.32106.194
    • Windows 10 SDK version 10.0.19041.0
    ✗ Visual Studio is missing necessary components. Please re-run the Visual Studio installer for the "Desktop development with C++" workload, and include these components:
        MSVC v142 - VS 2019 C++ x64/x86 build tools
         - If there are multiple build tool versions available, install the latest
        C++ CMake tools for Windows
        Windows 10 SDK

[✓] Android Studio (version 4.2)
    • Android Studio at R:\Rob\Programs\Android\Studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6842174)

[✓] VS Code, 64-bit edition (version 1.66.0)
    • VS Code at C:\Program Files\Microsoft VS Code
    • Flutter extension version 3.38.1

[✓] Connected device (3 available)
    • Mi Max 3 (mobile) • ad0fda83 • android-arm64  • Android 10 (API 29)
    • Windows (desktop) • windows  • windows-x64    • Microsoft Windows [Version 10.0.22000.469]
    • Edge (web)        • edge     • web-javascript • Microsoft Edge 98.0.1108.56

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 3 categories.

Flutter Application Metadata
Type: app
Version: 1.0.0+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: 097d3313d8e2c7f901932d63e537c1acefb87800
Plugins
flutter_contacts-1.1.3
flutter_secure_storage-5.0.2
flutter_secure_storage_linux-1.1.0
flutter_secure_storage_macos-1.1.0
flutter_secure_storage_web-1.0.2
flutter_secure_storage_windows-1.1.2
fluttertoast-8.0.9
google_mobile_ads-1.1.0
path_provider-2.0.9
path_provider_android-2.0.12
path_provider_ios-2.0.8
path_provider_linux-2.1.5
path_provider_macos-2.0.5
path_provider_windows-2.0.5

Closing as duplicate of https://github.com/flutter/flutter/issues/66224


Grade School False Positives

I believe we could better catch false positives on this exercise with inspect.
https://docs.python.org/2/library/inspect.html#inspect.isgenerator

Do you have specific examples of false positives that the current test suite isn't catching?

The problem is that if the user uses a structure that is unordered, they may get lucky and pass the suite.
https://github.com/exercism/xpython/issues/151
I wrote this as a reminder to investigate if inspect may allow us to better handle the case. Right now we check if they use a generator, sequence, or a class which implements the python magic method __reversed__ to check if they are using an ordered structure.

Ah, right. That makes sense. Thanks for clarifying.

Is there a real difference between for example isinstance(foo, types.GeneratorType) or inspect.isgenerator(foo)?

I looked into the source and found that inspect.isgenerator(foo) actually calls isinstance(foo, types.GeneratorType)
https://hg.python.org/cpython/file/2.7/Lib/inspect.py#l179


How to yield splash request when using seed file

Hello there,
I had a spider, which works fine for now, but since requirements are growing I looked at frontera to have millions of page to download. Since my urls are pre-defined, I have put them in a seed file. The pages also need splash rendering. I can't figure out, how to handle the splash request using the seed file. The start_requests is not getting called when using seed.
Here is my code:
`
import scrapy
from scrapy_splash import SplashRequest
from nothingbutsales.items import FootasylumItem
from scrapy.loader import ItemLoader
class FootasylumSpider(scrapy.Spider):
name = 'footasylum'
http_user = 'user'
http_pass = 'userpass'
# result = {}
# with open('/home/shopforsales/public_html/datafeed/beforescrape/footasylum.csv', 'rb') as f:
#     reader = csv.DictReader(f, delimiter=';')
#     for row in reader:
#         for column, value in row.iteritems():
#             result.setdefault(column, []).append(value)

# allowed_domains = ['www.footasylum.com']
# start_urls = result['merchant_deep_link']

custom_settings = {
    'LOG_FILE': '/tmp/' + name + '.log',
    'SEEDS_SOURCE': '/home/shopforsales/public_html/datafeed/beforescrape/seeds.txt'
}

def __init__(self, *args, **kwargs):
    super(FootasylumSpider, self).__init__(*args, **kwargs)

def start_requests(self):
    for url in self.start_urls:
        yield SplashRequest(url, self.parse)

def parse(self, response):
    print response.url
    price = response.css('li#priceFrm')
    rrp_price = price.css('span.wasprice::text').extract_first()
    print(rrp_price)

    if rrp_price is not None:
        loader = ItemLoader(item=FootasylumItem(), response=response)
        loader.add_value('rrp_price', rrp_price)
        loader.add_value('merchant_deep_link', response.url)
        loader.add_css('product_name', 'h1[itemprop="name"]')
        loader.add_css('brand', 'div[itemprop="brand"]')
        loader.add_css('breadcrumb', 'div.breadcrumb span a')
        loader.add_css('colour', 'title')
        loader.add_css('description', 'li#prod_descr')
        size_loader = loader.nested_xpath('//*[@id="uloption2"]')
        size_loader.add_css('sizes', 'span.sizevariant')
        size_loader.add_css('sizes', 'span.sizevarianttooltip')

        return loader.load_item()

`
Hope you can help me.

Frontera calls the make_requests_from_url method of the spider to create a Request from the seeds provided. have a look here. https://github.com/scrapinghub/frontera/blob/master/frontera/contrib/scrapy/middlewares/seeds/init.py#L18.
You could do:
class YourSpider(Spider):
     def make_requests_from_url(self, url):
          return SplashRequest(url)

     def parse(self, response):
         ...
         ...

Thank you very much. I will try and update you in the evening.
In my scenario, I need to read 150k urls from a csv/db table and have it in the seed.
So I will have to read the db, save it in the seed.
Is there anyway I can do it in a faster?
Thanks.

Hi @voith
I tested your solution, but unfortunately it only scrape the first url. Any idea please?

Hi @MuhammadRahman-awin
I'm not very sure if splash works very well with frontera. I'm not a splash user myself, so I'm not very familiar with the request/response cycle of splash.
If you have the desire to debug frontera to get your splash code working than I could point you to a couple of places in the code for possible suspects.

https://github.com/scrapinghub/frontera/blob/master/frontera/contrib/requests/converters.py.
Frontera converts scrapy's builtin Request to its custom FrontierRequest and return scrapy's default Response to the spider. Frontera assumes that the user will always use the default Request and Response.

Check if your requests are being scheduled. Add a debugger in this method https://github.com/scrapinghub/frontera/blob/master/frontera/contrib/scrapy/schedulers/frontier.py#L91. See If you get all you requests here.
Btw, which backend are you using? Can you see your requests added to your datastore?

Also have a look at https://github.com/scrapinghub/frontera/issues/232

Further to my investigation, I found the SchedulerSpiderMiddleware does not recognise the splash request.
@voith  your snippet actually works when I comment out this middleware and the Frontera scheduler
# 'frontera.contrib.scrapy.middlewares.schedulers.SchedulerSpiderMiddleware': 1000,
and
# SCHEDULER = 'frontera.contrib.scrapy.schedulers.frontier.FronteraScheduler'
Any suggestion please? Also what is this scheduler and why do I need to use it?
Thanks a lot

commenting lines :
'frontera.contrib.scrapy.middlewares.schedulers.SchedulerSpiderMiddleware':
1000,
and
SCHEDULER = 'frontera.contrib.scrapy.schedulers.frontier.
FronteraScheduler'
is a good as using plain scrapy without frontera
On Mon, Nov 6, 2017 at 3:33 AM, MuhammadRahman-awin <
<EMAIL_ADDRESS>wrote:

Further to my investigation, I found the SchedulerSpiderMiddleware does
not recognise the splash request.
@voith https://github.com/voith your snippet actually works when I
comment out this middleware and the Frontera scheduler
'frontera.contrib.scrapy.middlewares.schedulers.SchedulerSpiderMiddleware':
1000,
and
SCHEDULER = 'frontera.contrib.scrapy.schedulers.frontier.
FronteraScheduler'
Any suggestion please? Also what is this scheduler and why do I need to
use it?
Thanks a lot
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scrapinghub/frontera/issues/300#issuecomment-342009543,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHyuuUTcBVVQMV7ipNd7YcS36-IgT_Llks5szjCmgaJpZM4QMG8O
.

I think first step for @m-usman-dar is make Scrapy work without Frontera with Splash using SplashMiddleware.

submit a PR. we can discuss there if you think there's something wrong with frontera


Request for PHP Pages

Hey Buddy,
Can you please provide me the pages of PHP that you showed in Online demo.
I want to use functionality of "Try your own face"
When i submit custom image that action goes on upload.php
can you please provide me upload.php page ?
Thank You.

No sorry.

Its okay..
can you please explain what exactly did you do.
I mean co-ordination of php and running matlab run.m
Please at least tell me a steps to do.
Little help appreciated.

The online demo now uses the python version. It runs the script.

Thank you very much Mate.


fix: MD032/blanks-around-lists

Lists should be surrounded by blank lines

@nschonni : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Stream-style Serialization Follow-ups

List of follow-ups for the work on using stream-style serialization in azure-core.

[x] Add licensing for shaded code.
[x] Determine guidance on how to prevent changes to shaded code (aside from compiler suppressions).
[x] Investigate merging azure-core-serializeradapter-jackson into azure-core-serializer-json-jackson.
[x] Add logging.

[x] When JSON properties and structures (arrays and objects) are skipped/ignored.

[x] Determine final name for JsonCapable.
[x] Determine what package will contain the serialization interfaces.
[x] Ensure recursive structures do not allow infinite recursion.
[x] Determine which APIs in JsonUtils should be merged back into JsonWriter or JsonReader.
[x] Support reading full object from root scope.

Closing as complete as azure-json has shipped GA


iOS 9.3.2 Broke Scanner?

I have some clients reporting The Scanner is no longer working on their phones. This seems to fall around the same time the iOS 9.3.2 update came out. Anyone else getting any issues on iOS 9.3.2 devices?
Thanks

same for me with iphone 6 with ios v8.4.

Nevermind. It works fine. Apologies. Feel free to remove.


offering setup.py

Hello, I was wondering if it is possible for you to provide setup.py to automatically install the package.
Thanks!

The setup.py is still not there


providers: rename vagrant-virtualbox to virtualbox

This allows us to avoid using specific oem-ids for vagrant provisioned
machines and use more ids

there doesn't seem to be consensus on this rename, and it is now stale. if we still consider this a problem with the rust rewrite, we should open a new pr.

There is a consensus, I just haven't written it up or implemented it.  I'll reopen on the other project.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_bd52d6a1-7bf2-49fd-b0b2-c782ab3f1c10/pull/1 is merged into live branch.
Auto archive content to https://github.com/VSC-Service-Account/docs-archive-test-target_bd52d6a1-7bf2-49fd-b0b2-c782ab3f1c10.git

Docs Build status updates of commit 62138ff:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-ServiceAccountCannotAccessRepository] Your repository https://github.com/VSC-Service-Account/docs-archive-test-source has enabled Docs Build service account. It seems that at least one of the accounts, e2ebd4, does not have Read permission to the repo. Please ask your repo admin to grant correct permission to service accounts (refer to this document: https://review.docs.microsoft.com/en-us/engineering/projects/ops/engdocs/how-to-grant-service-account-permission-in-your-repository?branch=master#ops-e2e-ppe) and then retry the build. Don't know who to contact? Refer https://repos.opensource.microsoft.com/VSC-Service-Account/repos/docs-archive-test-source/permissions/ to get admin list of the repo if the repo is in one of the organizations Microsoft owns.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Provide capability for a ballerina service to be run as a windows service

$subject to facilitate Ballerina services to run as a windows service.

Will not implement this.


Improve stack traces of errors

The stack traces currently don't help with the debugging, because the package doesn't use await/async but Promise chains. For example, the error below doesn't say where in code the problem occurred. I think we can safely move to await/async, or at least have two builds - one for browsers and one for Node.js.
2019-02-15T15:46:18.433Z ERROR: PhantomCrawler: Unhandled exception (error details: type=invalid-value, statusCode=400)
2019-02-15T15:46:18.435Z   ApifyClientError: Invalid value provided: id is not allowed by the schema
2019-02-15T15:46:18.437Z     at exports.newApifyClientErrorFromResponse (/home/myuser/node_modules/apify-client/build/utils.js:87:12)
2019-02-15T15:46:18.439Z     at exports.requestPromise (/home/myuser/node_modules/apify-client/build/utils.js:158:19)
2019-02-15T15:46:18.440Z     at process._tickCallback (internal/process/next_tick.js:68:7)

Fixed in v1

Fixed in v1


Support unicode identifiers

Uses regexpu-core to compile the beyond-BMP \u into ES5-compatible regexps without having to include the big surrogate pair matching regexps.
Because the own property order is significant, split the definition of selectors and use for .. in to copy additional selectors over.
Only class and id selectors are supported.
Fixes using things such as emoji in class names, as might be generated by css-loader when the localIdentName option includes [emoji].

[Current coverage][cc-pull] is 100%

No coverage report found for master at db76c76.

Powered by Codecov. Last updated by [db76c76...17379f6][cc-compare]
[cc-base-branch]: https://codecov.io/gh/css-modules/css-selector-tokenizer/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/css-modules/css-selector-tokenizer/compare/db76c764f9ca266e90c1194fd01c6e1ad9b3d56d...17379f6d08b0ec66567701d1f558bc0646ca6fdb
[cc-pull]: https://codecov.io/gh/css-modules/css-selector-tokenizer/pull/14?src=pr


App crashes when network down

Error if trying to get weather and the network is down. Need to implement debugging, as well as graceful, non crashing exception handling

checked in change around the MSweather that was crashing the code. can add more debugging checks when needed. SunRiseSet had debugging installed around it


crates.io release?

Could you release the code from master?

@medek You either need to add owners on crates or do the release yourself

Yeah my bad. Computer was out of commission for a bit new release published


--input_file requires an available cluster, does not support yaml files with multiple resources, does not give errors

What happened:
I wanted to convert ingresses in a multi-yaml file usin --input_file, without access to the cluster.
./ingress2gateway print --input_file harbor.yaml
resulted in
Error: failed to create client: Get "http://localhost:6443/api?timeout=32s": dial tcp <IP_ADDRESS>:6443: connect: connection refused
because the cluster is airgapped and unavailable for my laptop. With connection to the cluster, it returns
No resources found in default namespace.
What you expected to happen:
ingress yamls are converted to httproutes.
How to reproduce it (as minimally and precisely as possible):
helm template harbor oci://registry-1.docker.io/bitnamicharts/harbor --set exposureType=ingress -set ingress.core.ingressClassName=nginx -n harbor > harbor.yaml
./ingress2gateway print --input_file harbor.yaml
the above gives no meaningful output, but should either give an error or the expected output.

related to https://github.com/kubernetes-sigs/ingress2gateway/pull/78

Should be fixed by https://github.com/kubernetes-sigs/ingress2gateway/pull/128
Now for reading local files the k8sclient creation is skipped.

@simonfelding I believe this issue is fixed now.
Can you confirm?

closing this as fixed. Feel free to reopen if needed.

@LiorLieberman I have still this error with 0.2.0:

My only wish is to use the cli offline, on our gitops repository

Hi @davinkevin, I guess the resources in your ingress.yaml file do not specify a namespace (and if they do, it isn't the default one). Please use -A (--all-namespaces) flag

Thank you for your answer @levikobi . Result is different but not really better.

My file is just this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: podinfo.ingress.k8s.local
spec:
  rules:
    - host: podinfo.ingress.k8s.local
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: podinfo
                port:
                  name: http

@davinkevin are you sure you're using release 0.2?
I can't seem to reproduce the error you're showing here TBH.. Neither on main

I would say yes 🤔

Hey @davinkevin the reason is that you did not specify ingressClassName in spec. So no provider pick it up.
Also, I think there is a bug when using port.name. Can you try to add ingressClassName and change port.name to port.number and report back if it works?

Hello @LiorLieberman,
So, I've first tried with a specific ingressClassName and the change you mentioned:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: podinfo.ingress.k8s.local
spec:
  ingressClassName: nginx
  rules:
    - host: podinfo.ingress.k8s.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: podinfo
                port:
                  number: 1234

And it's working! The result is:
ingress2gateway print --input_file=ingress.yaml --providers=ingress-nginx -A
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  creationTimestamp: null
  name: nginx
spec:
  gatewayClassName: nginx
  listeners:
  - hostname: podinfo.ingress.k8s.local
    name: podinfo-ingress-k8s-local-http
    port: 80
    protocol: HTTP
status: {}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  creationTimestamp: null
  name: podinfo.ingress.k8s.local-podinfo-ingress-k8s-local
spec:
  hostnames:
  - podinfo.ingress.k8s.local
  parentRefs:
  - name: nginx
  rules:
  - backendRefs:
    - name: podinfo
      port: 1234
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents: []

If I use the named port, I indeed have an error:

So, the main remark I could do is, for newcomers, it's really a shame to not be able to use the tool on bare ingress files.
Maybe a generic provider to manage ingress if all others didn't pick anything?

I could be convinced the idea is good.
Would you like to go ahead and contribute such a thing?

Good, unfortunately, I don't think I'll get time to provide a quality PR for that support… #Sorry
However, I keep that in mind, in case I have more time for it, but I'm sure something we'll have done it before me 😅


work on quat and matrix math - deleted multiple copies of raymath.h …

…causing issues
-isystem doesn't seem to work for me, is there a specific reason to use it?
its worth noting for quat eular operations that its important to use a ZYX matrix rotation order.

@chriscamacho Thanks for the improvement, I'll review it now.


typing error: computed

i use<EMAIL_ADDRESS>and<EMAIL_ADDRESS>there is reference error while typing computed.
it's not show but i forgot what version of vue that i used before, but after i upgrade this error show up.

Type 'ComputedRef<string | undefined>' is not assignable to type 'MaybeRef<string> | undefined'.
  Type 'ComputedRef<string | undefined>' is not assignable to type 'Ref<string>'.ts(2322)

<script setup lang="ts">
import { computed } from "vue";
import { useHead } from "@vueuse/head";

const data = reactive<IMangaResult>({});

useHead({
  title: computed(() => data.title),
});
</script>

I have the same situation why I can't use it like this


Suggestion: Menu placement

Have the menu options slide out to the left of the 3 bar menu button, rather than the current side bar, in order to prevent overlapping the 3 streams.

f75382d should minimize this issue.
The menu component is summoned by clicking three dots and closed by clicking anywhere outside. The menu width and height are set by min-content which should constrict the dimensions to only what's necessary.


Adapt "Adjustment" pattern into Distributions

From the meeting:
We want to avoid two scenarios:

A partner shows up to pick up a distribution and needs more, and an additional distribution needs to be created
A partner shows up and needs less than originally promised, and the distribution needs to be trashed and rebuilt

We already use adjustments for inventories, we could probably adapt the adjustment model to also work with distributions maybe? Or at least use the same pattern (and refactor to a concern?)

This was closed with #299


Could  you release the cityscape video dataset?

That's so kind of you if you release that..the registration of the cityscape website requires few days...

Sorry. It's difficult. First, the size of the video dataset is huge, so it is hard to share it with Dropbox or BaiduPan. Second, the video dataset is available upon request on the official homepage.


[Web][app_check]: hot restarting / hot reloading throws Error: [app-check/already-activated] AppCheck: You are trying to activate AppCheck for FirebaseApp [DEFAULT], while it is already activated. AppCheck can only be activated once.

Discussed in https://github.com/FirebaseExtended/flutterfire/discussions/8017

Originally posted by misteroak February  4, 2022
Hi,
I am likely missing something here, but couldn't find an answer online. I've been following the instructions to add AppCheck to my Flutter Web app. It's working fine on first load, but the moment I hit hot reload or hot restart (using Vs Code) I get the following error:
Error: [app-check/already-activated] AppCheck: You are trying to activate AppCheck for FirebaseApp [DEFAULT], while it is already activated. AppCheck can only be activated once. .
Here's my initialization code:
void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await Firebase.initializeApp(
    options: DefaultFirebaseOptions.currentPlatform,
  );

  await FirebaseAppCheck.instance.activate(
    webRecaptchaSiteKey: READCTED-TOKEN,
  );

  BlocOverrides.runZoned(
    () {
      runApp(MyApp());
    },
    blocObserver: AppBlocObserver(),
  );
}

What am I doing wrong?

@misteroak
Can you try the plugin example and see if you still get same error using it ?

Hey, thanks for you response!
Yes, sadly, I'm getting the exact same error using the example. Cloned it as is and ran it in debug mode. I see the main app screen, but once I hit Restart, I get the following error in the console (all in VS Code):
Restarted application in 174ms.
Error: [app-check/already-activated] AppCheck: You are trying to activate AppCheck for FirebaseApp [DEFAULT], while it is already activated. AppCheck can only be activated once. .
    at Object.throw_ [as throw] (http://localhost:52695/dart_sdk.js:5063:11)
at http://localhost:52695/packages/firebase_core/src/internals.dart.lib.js:81:63
    at _RootZone.runUnary (http://localhost:52695/dart_sdk.js:40062:59)
    at _FutureListener.catchError.handleError (http://localhost:52695/dart_sdk.js:34996:33)
    at handleError (http://localhost:52695/dart_sdk.js:35567:51)
    at Function._propagateToListeners (http://localhost:52695/dart_sdk.js:35593:17)
    at _Future.new.[_completeError] (http://localhost:52695/dart_sdk.js:35443:23)
    at async._AsyncCallbackEntry.new.callback (http://localhost:52695/dart_sdk.js:35479:31)
    at Object._microtaskLoop (http://localhost:52695/dart_sdk.js:40330:13)
    at _startMicrotaskLoop (http://localhost:52695/dart_sdk.js:40336:13)
    at http://localhost:52695/dart_sdk.js:35811:9

@misteroak
Thanks for the update. I tried to replicate it locally, but am hitting https://github.com/FirebaseExtended/flutterfire/issues/6310, so can't really verify the behavior you are seeing.
Also, looking at the linked issue, it may be happening due to the fact that, the app_check plugin is early access preview plugin and may have open issues that the team may be working on to fix. As a side note, you can also follow-up in the linked issue to know the current status of plugin working on web.
Labeling this for now based on the error log.

Any update on this? Really makes web dev painful when you can't hot reload?

Any update as you can't seem to turn off App Check on Firebase once you enable it so we are stuck unable to hot restart?

+1

+1 Fixing this would help my dev experience a lot. Thanks!

I just put the initialization of App Check in a try catch block, which makes it work for dev, but that can't be the official solution, especially since the firebase initializeApp has no such problems.

+1

Please help to resolve, I have the problem also when hot reload.

+1

Instead of commenting +1 or same issue or me too, please upvote original issue, that helps team to prioritize issue.

/cc @russellwheatley

I've just tested this against latest on master branch which now uses web v9 JS SDK and it is no longer an issue.


Reload a a specific module

Add a argument to /ct load which can be used to reload a specific module.

better suggestion: store hashes of files, then compare hashes, only reload the ones that changed

This would be a non-trivial change, but I can see the benefit. If I were to add this it would be in a release after 2.0.

other method: store hashes of files, then compare hashes, only reload the ones that changed
ofc this is just me predicting the https://xyproblem.info/

This doesn't make sense; /ct load causes side effects. Plus you can't just reload a single file, since all of the files are loaded together.

what about
/ct load: reload all modules that changed, tell the user which ones it's reloading, exclude data files if possible
/ct load [module]: reload only that module
/ct load all: reload all modules

If you have a module that is so slow that you want to avoid loading it then you should just delete it. I will absolutely not implement file watching.

If you have a module that is so slow that you want to avoid loading it then you should just delete it.

isn't that the point of the original issue?

I will absolutely not implement file watching.

i was thinking of not doing it realtime.

Yeah after some research this isn't really possible without some Rhino hacking, of which we already had to do plenty of to even get it to work in the first place. Plus it's been over a year since I've really worked with Rhino, and I'm not trying to break that streak any time soon.


Add more details for authenticating with different users (AzureAD and external)

For more info: https://docs.microsoft.com/answers/answers/398089/view.html
Please add some more details for authenticating with different users (AzureAD and external). Specifically when it comes to using the different endpoints, https://login.microsoftonline.com/common/v2.0 instead of https://sts.windows.net/tenant_id/v2.0
Changing the Issuer URL within your App service's authentication:
https://login.microsoftonline.com/common/v2.0
-This allows any user to login without the need to be added to your AzureAD tenant.
https://sts.windows.net/tenant_id/v2.0
-Any user can still log in using the default Issuer URL. However, external users will need to be added to your AzureAD tenant.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ce623170-1377-b01b-b565-b954e25e2f7d
Version Independent ID: 575b95e7-302f-4875-bfd0-876851e18a54
Content: Tutorial - Add authentication to a web app on Azure App Service - Azure App Service
Content Source: articles/app-service/scenario-secure-app-authentication-app-service.md
Service: app-service-web
GitHub Login: @rwike77
Microsoft Alias: ryanwi

@JamesTran-MSFT
Thanks for your feedback! We will investigate and update as appropriate.

We have raised a PR on adding a note on this ("details on authenticating with different users (AzureAD and external)")- once the PR is merged the doc would reflect the changes. We will close this for now.


about Json Parsing

i like this json library ,it's very easy to use
https://github.com/lingoer/SwiftyJSON
could you support this ?

@harshcs can you describe the implementation that you did.
I am newbie to swift and Alamofire


[13pt] Remove NWM streamlines that are from neighboring HUC8s

During clip_vectors_to_wbd.py, there are some erroneous NWM stream segments that are captured from neighboring HUC8s. We need to identify and remove these segments so that they are not part of a GMS branch or modeled by HAND.

Here is where gms/run_by_unit.sh does the clip.

Screenshot of the issue in HUC 07080101

Also check out this HUC: 01090004. I think it may be happening there too, but I haven't confirmed yet.


Enable devcontainers

This provisioning option offers the alternative to create Sandbox environments on GitHub Cloud. This feature can be limited in HW resources to some users based on their GH account.

/retest

/retest

/approve


Crash in GDTCORClock.m

[REQUIRED] Step 1: Describe your environment

Xcode version: 11.3
Firebase SDK version: 6.16.0
Firebase Component: RemoteConfig
Component version: 6.16.0
Installation method: CocoaPods

[REQUIRED] Step 2: Describe the problem
Steps to reproduce:
Crash on iOS App launch with Firebase pods installed in the iOS app.
Crash happens in all most all OS versions and devices.
EXCEPTION - SIGSEGV
Relevant Code:
We have got crash reports , could not reproduce it on code.
New relic Crash report:
// TODO(you): code here to reproduce the problem

Crash, GDTCORClock.m line 86 in [GDTCORClock init]

Please share the full stack trace of the crash.

Cleartrip-iOS-crashreport-02-11-2020_0639+0530.crash.zip

Here's the thread of interest:
Thread 13 name:
Thread 13 Crashed:
0   libobjc.A.dylib                	 0x00000001b51d2088 objc_msgSend (libobjc.A.dylib)
1   CoreFoundation                 	 0x00000001b5387684 -[__NSDictionaryM objectForKey:] (CoreFoundation)
2   hookkeyboard.dylib             	 0x0000000106029c70 0x106000000 + 171120
3   hookkeyboard.dylib             	 0x000000010600de30 0x106000000 + 56880
4   CoreFoundation                 	 0x00000001b54c9b4c CFTimeZoneCopySystem (CoreFoundation)
5   Cleartrip                      	 0x0000000102f6552c -[GDTCORClock init] (GDTCORClock.m:86)
6   Cleartrip                      	 0x0000000102f6557c +[GDTCORClock snapshot] (GDTCORClock.m:94)
7   Cleartrip                      	 0x0000000102f655a0 +[GDTCORClock clockSnapshotInTheFuture:] (GDTCORClock.m:99)
8   Cleartrip                      	 0x0000000102f6bc4c -[GDTCORUploadPackage initWithTarget:] (GDTCORUploadPackage.m:43)
9   Cleartrip                      	 0x0000000102f6d334 -[GDTCCTPrioritizer uploadPackageWithConditions:] (GDTCCTPrioritizer.m:61)
10  Cleartrip                      	 0x0000000102f6b218 __52-[GDTCORUploadCoordinator uploadTargets:conditions:]_block_invoke (GDTCORUploadCoordinator.m:118)
11  libdispatch.dylib              	 0x00000001b517a610 _dispatch_call_block_and_release (libdispatch.dylib)
12  libdispatch.dylib              	 0x00000001b517b184 _dispatch_client_callout (libdispatch.dylib)
13  libdispatch.dylib              	 0x00000001b5127404 _dispatch_lane_serial_drain$VARIANT$mp (libdispatch.dylib)
14  libdispatch.dylib              	 0x00000001b5127df8 _dispatch_lane_invoke$VARIANT$mp (libdispatch.dylib)
15  libdispatch.dylib              	 0x00000001b5131314 _dispatch_workloop_worker_thread (libdispatch.dylib)
16  libsystem_pthread.dylib        	 0x00000001b51cab88 _pthread_wqthread (libsystem_pthread.dylib)
17  libsystem_pthread.dylib        	 0x00000001b51cd760 start_wqthread (libsystem_pthread.dylib)

Here is the init method that supposedly is causing the crash. It doesn't contain anything immediately suspicious.
It looks like there's two symbols that were not properly symbolicated that may not be part of your app's code, originating from hookkeyboard.dylib.
2   hookkeyboard.dylib             	 0x0000000106029c70 0x106000000 + 171120
3   hookkeyboard.dylib             	 0x000000010600de30 0x106000000 + 56880

Is this app running on a jailbroken device?

Oh , yeah. It looks like. Thanks for the bringing to this attention.
I guess our crash reports are specific to Jailbroken devices only.
Will close the thread.
Thanks for helping on this :)

No problem!


Generalise to Python (and Docker?)

The arguments around R and R packages will be similar for Python and Docker, though their repos (e.g. PyPi and Dockerhub) are run somewhat differently to CRAN as I understand it.
Can this therefore be generalised, but with a few R/Python specific chunks mixed in there.
Not to overcomplicate things but I just discovered RMarkdown parameters; I could just imagine having a document here where you can select Python or R at the beginning and the entire document adjusts accordingly ...

I agree with you, I wouldn't want to over-generalise it though. I think it would be good to have something that goes:

Here is guidance for open tooling and code sharing
Here is guidance for R
Here is guidance for Python
Here is guidance for Docker

However, I wouldn't want the NHS-R community to produce guidelines for using Python or Docker, I'd like help from others to do this


With SQLStandardAccessControl all users can see all schemas and all tables

Even listing schemas and tables can reveal information.
It would be better if users can only see:

schemas they own (either directly or by role)
tables they have any access permission to (any of SELECT, INSERT, DELETE)

The methods are already present in SQLBaseAccessControl, they just need to be implemented.
Catalog access is handled outside of the Hive connector (file based system access control)
It's not clear for Hive does.

If people agree with the above, I'll do a PR.

This isn’t implemented because it’s very expensive, as the metastore has no bulk access for permission data. SHOW TABLES would have to do a fetch for each table.
Validating the schema is more doable as there are usually few schemas, plus users typically only access a single schema at a time.

I see. We could simplify this such you can't any tables if you do not have access to the schema. Then it's a schema check only in both cases.

Just this?
diff --git a/presto-hive/src/main/java/io/prestosql/plugin/hive/security/SqlStandardAccessControl.java b/presto-hive/src/main/java/io/prestosql/plugin/hive/security/SqlStandardAccessControl.java
index ded7662a4a..7802eea038 100644
--- a/presto-hive/src/main/java/io/prestosql/plugin/hive/security/SqlStandardAccessControl.java
+++ b/presto-hive/src/main/java/io/prestosql/plugin/hive/security/SqlStandardAccessControl.java
@@ -83,9 +83,11 @@ import static io.prestosql.spi.security.AccessDeniedException.denyShowColumns;
 import static io.prestosql.spi.security.AccessDeniedException.denyShowCreateSchema;
 import static io.prestosql.spi.security.AccessDeniedException.denyShowCreateTable;
 import static io.prestosql.spi.security.AccessDeniedException.denyShowRoles;
+import static io.prestosql.spi.security.AccessDeniedException.denyShowTables;
 import static io.prestosql.spi.security.PrincipalType.ROLE;
 import static io.prestosql.spi.security.PrincipalType.USER;
 import static java.util.Objects.requireNonNull;
+import static java.util.stream.Collectors.groupingBy;
 import static java.util.stream.Collectors.toSet;

 public class SqlStandardAccessControl
@@ -202,12 +204,19 @@ public class SqlStandardAccessControl
     @Override
     public void checkCanShowTables(ConnectorSecurityContext context, String schemaName)
     {
+        if (!isDatabaseOwner(context, schemaName)) {
+            denyShowTables(schemaName);
+        }
     }

     @Override
     public Set<SchemaTableName> filterTables(ConnectorSecurityContext context, Set<SchemaTableName> tableNames)
     {
-        return tableNames;
+        return tableNames.stream()
+                .collect(groupingBy(SchemaTableName::getSchemaName, toSet())).entrySet().stream()
+                .filter(schema -> isDatabaseOwner(context, schema.getKey()))
+                .flatMap(schema -> schema.getValue().stream())
+                .collect(toSet());
     }

     @Override

Note: I haven't tried, this is for demonstration what I mean.

Lemme just do a PR with tests, and then we can discuss there. It'll be quick.

Validating the schema is more doable as there are usually few schemas, plus users typically only access a single schema at a time.

In out case we'll have many schemas, and at least would want to hide tables in non-accessible schemas - see the PR, which does exactly that.

How does SQL standard Hive behave when user has a SELECT privilege to a table? I would guess that then schema (database) that encloses this table, is visible for user?  If so, then for every schema that is not directly granted to user, we need to check all tables. Notice, that same story is for columns. Schema might be visible, whenever user has access to any column under this schema. That is so expensive to check.

Hmmm... Interesting! SELECT ... FROM TABLE s.t still works even when a user has no access to schema s. (In Presto)
The problem is really twofolds:

There's no GRANT/REVOKE on schemas (the Hivemetastore seems to have it all, it's not implemented anywhere). So at the schema level it's all or nothing.
There's not bulk API to check permissions for HiveObjects.

I'll drop this line of development for now.
For our needs we'll likely either have a patched Hivemetastore that does not kind of filtering for us or that has a bulk api for access checks, along with a metastore proxy that tracks other things for us - such as usage, lineage, etc.
Closing this for now.

Notice that it is quite easy to filter schemas with file based system access control. You could use it along with SQL standard. It is detached from HMS, but it is refreshable so you can write on your own something that could generate file with rules.


Update docs for V0.8.0

Installation guides are not updated because we need a new tag.

Python SDK for milvus v0.8.0 is v0.2.10

Go sdk for milvus v0.8.0 is  v0.3.0

Java SDK: 0.7.0


ecdsa ethereum compatible

Hi,
Does the signing function compatible with ethereum? it looks like it uses sha256 instead of keccak256:
https://github.com/ETCDEVTeam/libsecp256k1-rs/blob/master/src/lib.rs#L302
Is it possible to get the hashing mechanism out of the signing or replace sha256 with keccak256?
Thanks,
Elichai.

Just writing down the answer from gitter, in case anyone else's looking for it. :)
Yes, it is ethereum-compatible. HMAC-SHA256 is used in that line for nonce generation, as defined in RFC 6979.


BackChannel Logout is not handled when session termination

In the WSO2 identity server, we can terminate sessions from,

Session termination Rest API[1]
Session limit handler[2]
In both of them, the identity server calls the SessionManagementService to terminate the user's session. However, as per the implementation of the SessionManagementService, when we publish the SESSION_TERMINATE event here we are passing the HTTPServletRequest as null.

    public static void publishSessionEvent(String sessionId, HttpServletRequest request, AuthenticationContext
            context, SessionContext sessionContext, AuthenticatedUser user, String status) {

Check below,
    private void terminateSession(SessionContext sessionContext, String sessionId) {

        if (FrameworkServiceDataHolder.getInstance().getAuthnDataPublisherProxy() != null && FrameworkServiceDataHolder
                .getInstance().getAuthnDataPublisherProxy().isEnabled(null) && sessionContext != null) {

            Object authenticatedUserObj = sessionContext.getProperty(FrameworkConstants.AUTHENTICATED_USER);
            AuthenticatedUser authenticatedUser = new AuthenticatedUser();
            if (authenticatedUserObj != null) {
                authenticatedUser = (AuthenticatedUser) authenticatedUserObj;
            }
            FrameworkUtils.publishSessionEvent(sessionId, null, null, sessionContext, authenticatedUser,
                    FrameworkConstants.AnalyticsAttributes.SESSION_TERMINATE);
        }
        SessionContextCache.getInstance().clearCacheEntry(sessionId);
    }

Because of this, the backchannel logouts are not triggered. The reason is we are expecting the HttpServaletRequest to be not null in the SAMLLogoutHandler[5] and in the OIDCLogoutHandler[6] as well. Because the HttpServerletRequest is null the if conditions fail and the backchannel logouts are not initiated.
I have thought of resolving this and come up with several ways;
It seems that we may have to cache the required parameters from the aforementioned session limit handler and the session terminate rest API. After that, we need to read the values from the cache to correctly handle the scenario from each protocol component.
2. Another way is that we can introduce a different event such as FORCE_SESSION_TERMINATE and handle it separately.
Personally, I believe that the second way is better. In both ways, we need to correctly handle the protocol-specific cookies as well.
[1] - https://github.com/wso2/identity-api-user/tree/master/components/org.wso2.carbon.identity.api.user.session
[2] - https://github.com/wso2-extensions/identity-local-auth-basicauth/blob/62db1355210c1fe85cdf6c5bf17038f0ef2152bd/components/org.wso2.carbon.identity.application.authentication.handler.session/src/main/java/org/wso2/carbon/identity/application/authentication/handler/session/ActiveSessionsLimitHandler.java#L57
[3] - https://github.com/wso2/carbon-identity-framework/blob/master/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/UserSessionManagementService.java
[4] - https://github.com/wso2/carbon-identity-framework/blob/5e8dce309f2013dcea1a2d580de957a84bab1f83/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/services/SessionManagementService.java#L96
[5] - https://github.com/wso2-extensions/identity-inbound-auth-saml/blob/09ff84a3496f5407525ba6d1838acd06234abcf7/components/org.wso2.carbon.identity.sso.saml/src/main/java/org/wso2/carbon/identity/sso/saml/SAMLLogoutHandler.java#L57
[6] - https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/7dc33e832a13e9b2d6f0183c8e523957a033c6d0/components/org.wso2.carbon.identity.oidc.session/src/main/java/org/wso2/carbon/identity/oidc/session/handler/OIDCLogoutEventHandler.java#L55
The expectation is that once the user terminate the particular session, it also should handle backchannel logout scenarios.

This git issue

Fix back-channel logout issue upon session termination event for oidc and saml flows

Earlier back-channel logout depends on request object. Due to this when session termination even is triggered by REST api, this request object is not found. So back-channel logout is not getting triggered, when session termination event is fired.
So this PR address this issue by storing  samlssotokenId cookie value in the session context
In order to get the session context identifier, rather using the commonauthId cookie, it is passed as a property in the authentication result.


[Feature Request] Replace client app with ssh

Can you do something like serveo.net where I don't need to download any app and just use SSH?

It actually already works that way! But you need to download the private key for the tunnel. See the script here which shows the steps:
https://github.com/boringproxy/boringproxy/blob/6b36c256c32746e293a1bf7ce6ed3d45862501b7/scripts/tunnel.sh
It might be a bit out of the date, but shows the basic idea.
Note: I don't intend for boringproxy to become an alternative to free services like serveo.net. Those typically get overrun with spammers and phishers. The boringproxy demo site is for people to try it out without having to download themselves. If it becomes a problem I'll just shut it down. I might start a paid service at some point in the future though.

Note: I don't intend for boringproxy to become an alternative to free services like serveo.net. Those typically get overrun with spammers and phishers. The boringproxy demo site is for people to try it out without having to download themselves. If it becomes a problem I'll just shut it down. I might start a paid service at some point in the future though.

This is troubling. When can I expect this project to collect dust?
There's already ngrok.io, lynk.sh and more....

Note: I don't intend for boringproxy to become an alternative to free services like serveo.net. Those typically get overrun with spammers and phishers. The boringproxy demo site is for people to try it out without having to download themselves. If it becomes a problem I'll just shut it down. I might start a paid service at some point in the future though.

This is troubling. When can I expect this project to collect dust?
There's already ngrok.io, lynk.sh and more....

I don't intend for the project to collect dust any time soon, seeing as I use it every day for hosting my own things. I was specifically talking about the demo site. It's not intended for people to use for hosting. It's just to demo how the software works. If people abuse it I'll just turn it off rather than trying to moderate.

I don't intend for the project to collect dust any time soon, seeing as I use it every day for hosting my own things. I was specifically talking about the demo site. It's not intended for people to use for hosting. It's just to demo how the software works. If people abuse it I'll just turn it off rather than trying to moderate.

I don't intend for boringproxy to become an alternative to free services like serveo.net. Those typically get overrun with spammers and phishers.

I know, I just want to selfhost my own tunnel and connect it similar to serveo but with password. I'll check it out if it works

I don't intend for boringproxy to become an alternative to free services like serveo.net. Those typically get overrun with spammers and phishers.

I know, I just want to selfhost my own tunnel and connect it similar to serveo but with password. I'll check it out if it works

Gotcha. Since you prefer using your own SSH client, SirTunnel might be a better fit for you. It's very minimal/easy. It lets you control tunnels from the SSH client rather than a web UI on the server.

Gotcha. Since you prefer using your own SSH client, SirTunnel might be a better fit for you. It's very minimal/easy. It lets you control tunnels from the SSH client rather than a web UI on the server.

But can it forward tcp connection? Or is it http only?

But can it forward tcp connection? Or is it http only?

It's just a normal SSH tunnel, so it can do everything an SSH tunnel can. To forward TCP you'll need to have GatewayPorts=yes or GatewayPorts=clientspecified in your sshd_config. Make sure you understand the implications of setting that before doing it.

It's just a normal SSH tunnel, so it can do everything an SSH tunnel can. To forward TCP you'll need to have GatewayPorts=yes or GatewayPorts=clientspecified in your sshd_config. Make sure you understand the implications of setting that before doing it.

Alright thanks

Alright thanks

Going to go ahead and close this. Feel free to reach out if you have any more questions or issues.

Going to go ahead and close this. Feel free to reach out if you have any more questions or issues.


fix(html): support preload and prefetch of css

Description

Vite does not currently support the use of preload and prefetch tags for CSS files, despite the benefits of using these values. This issue is caused by a snippet of code in the HTML plugin (found here) that adds preload and prefetch tags with their href and location to a list called styleUrls, but it should only add these tags to links that include a rel attribute set to stylesheet. As a result, when this code snippet (found here) is executed to generate the link tags, the prefetch and preload tags are reduced down to their hrefs and subsequently eliminated, as they are the same as the href of the link tag with stylesheet.
Additional context
demo of the bug(build the project to see that the link tag is eliminated): https://stackblitz.com/edit/vitejs-vite-t1esnp?file=index.html

What is the purpose of this pull request?

[x] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[x] Read the Contributing Guidelines.
[x] Read the Pull Request Guidelines and follow the PR Title Convention.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

Closing pr since this seemingly doesn't address the issue.


Update coco_detection.py

Is this PR accidentally created?

Yes sorry apologies for the disturbance
On Tue, 19 May, 2020, 11:16 am Kai Chen<EMAIL_ADDRESS>wrote:

Is this PR accidentally created?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/open-mmlab/mmdetection/pull/2760#issuecomment-630594729,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKLHZOFIOUXC3AIEOEFMSD3RSIMNDANCNFSM4NEVJKWQ
.

Hi @Gugan0905！We are grateful for your efforts in helping improve this open-source project during your personal time.
Welcome to join OpenMMLab Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us ：https://discord.gg/UjgXkPWNqA
If you have a WeChat account，welcome to join our community on WeChat. You can add our assistant ：openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends：）
Thank you again for your contribution❤ @Gugan0905


Hide debug hover widget when sticky hover is disabled (fix #197463)

Closes #197463

@roblourens Could you please provide the steps to reproduce this bug?
I'm not sure how this bug occurs. Currently when DebugHoverWidget return result is not available,  DebugEditorContribution shows the editor hover.
https://github.com/microsoft/vscode/blob/7771124ebd6b7f6e60da14d744961b717b73282f/src/vs/workbench/contrib/debug/browser/debugEditorContribution.ts#L381-L385

All I know is

Disable editor.hover.sticky
Debug and activate the hover
See both hovers at the same time, sometimes


Is there an example for GPT series model?

If not, do we have plan to enable it?

GPT-J example is in our latest release


Latin unicode support

I've added support for Latin-1 Supplement Unicode characters (+ tests) and slightly modified the code to easily add new alphabets in the future.
I also noticed there was a .eslintrc.json file so I installed eslint as a devDependencies and added the linting to the test script.
Let me know if that works for you!

Hi @jonschlinkert!
Let me know if I need to change anything on this PR or if that's good for you! 😃

@AntonioVdlC sorry, missed your pr somehow. I'll review today, thanks!

What is the state of this pull request?