feat: idiomatic logging

NOTE: This is stacked on #100.

build(cargo): add log and pretty_env_log to dependencies
feat: add "idiomatic" rust logging

This introduces the oldlog feature to preserve the old logging style, but
default to using the more idiomatic logging macros with pretty_env_logger
(which should be easy to replace with something else).
This is partially to make the output more readable, and I was informed that the
logging format is kinda an idea that never took off.

NixOS vm integration tests also pass for this

I have no feelings for "oldllog", so you can drop support for it at will :) you people in DBC know best when "will" is.

I have no feelings for "oldllog", so you can drop support for it at will :) you people in DBC know best when "will" is.

I think the situation over here is basically as soon as possible, so we probably won't have to drag the feature around for that long... but for now, we still need it in some situations >_>

@cafkafk Mind resolving the conflict?

@cafkafk Mind resolving the conflict?

Will do :+1:

Pretty sure I messed that up

This is ready for review again


Anchors are huge

The anchor icons placed by the extension are pretty big on Perforce's site. For example: https://www.perforce.com/products/helix-alm/jira-integration

At least you can't miss them! That's pretty obnoxious, lol, I'll have to check out why it's doing that...

This should be fixed, after they approve it in the store. When it updates, it'll come through as version 1.3.0. Let me know if you run into any problems.

Works well now, thanks!


Allow to specify type comparator after using extracting on Object

Summary
Extend usingComparatorForElementFieldsWithType usage to extracting on Object
Example
Person joe = new Person("Joe", 25);
joe.setHeight(new BigDecimal("1.80"));

assertThat(joe).extracting("name", "height")
               .usingComparatorForElementFieldsWithType(new BigDecimalComparator(), BigDecimal.class)
               .containsExactly("Joe", new BigDecimal("1.8"));

While reviewing this, I now find that usingComparatorForElementFieldsWithType should have been usingComparatorForType as I think it clear that is you register a comparator for a given type, it's gonna be used whenever an instance of this type needs to be compared.
@PascalSchumacher thoughts ?

Well, usingComparatorForElementFieldsWithType method name does reflect current implementation behavior. Not sure if there might be cases when new behavior will not work instead of it (I mean, if there might be needed to have comparator registered for elements' fields, but not for elemetns themself). Agree that method name usingComparatorForType fits new behavior better.

I agree that
assertThat(joe).extracting("name", "height")
               .usingComparatorForType(new BigDecimalComparator(), BigDecimal.class)
               .containsExactly("Joe", new BigDecimal("1.8"));

reads better.
I'm not sure about deprecating usingComparatorForElementFieldsWithType. It think its name is well aligned with usingComparatorForElementFieldsWithNames. Both methods serve as a set-up for usingFieldByFieldElementComparator, usingElementComparatorOnFields, usingElementComparatorIgnoringFields and usingRecursiveFieldByFieldElementComparator.
Furthermore the use case is different, while usingComparatorForType works on the elements (see above), usingComparatorForElementFieldsWithType  works on the fields of the elements, e.g.:
public class Person {

    private final String name;

    public Person(String name) {
        this.name = name;
    }
}

@Test
public void test() {
    assertThat(new Person[] {new Person("John")})
        .usingComparatorForElementFieldsWithType(new AlwaysEqualStringComparator(), String.class)
        .usingFieldByFieldElementComparator()
        .containsExactly(new Person("Jane"));
}

I would like to figure out what is expected from usingComparatorForType.
Before that I would like to mention that when usingComparatorForElementFieldsWithType is called, comparator for element field by type is not getting applied until usingFieldByFieldElementComparator (or some other setting comparator method) which uses comparatorsForElementPropertyOrFieldTypes  is called, as by default elements are compared by equals.
For example:
This will not pass:
Person obiwan = new Person("Obi-Wan", new BigDecimal("1.820"));
Person obiwanClone = new Person("Obi-Wan", new BigDecimal("1.82"));`

assertThat(asList(obiwan))
  .usingComparatorForElementFieldsWithType(new BigDecimalComparator(), BigDecimal.class)
  .containsExactly(obiwanClone);

And this will:
assertThat(asList(obiwan))
  .usingComparatorForElementFieldsWithType(new BigDecimalComparator(), BigDecimal.class)
  .usingFieldByFieldElementComparator()
  .containsExactly(obiwanClone);

I can implement usingComparatorForType as:
public <T> SELF usingComparatorForType(Comparator<T> comparator, Class<T> type) {
if (iterables.getComparator() == null) {
  usingElementComparator(new ExtendedByTypeComparator(comparatorsForElementPropertyOrFieldTypes));
}

return usingComparatorForElementFieldsWithType(comparator, type);
}

So that next will pass:
assertThat(obiwan).extracting("name", "height")
  .usingComparatorForType(new BigDecimalComparator(), BigDecimal.class)
  .containsExactly("Obi-Wan", new BigDecimal("1.82"));

But probably there should be used different comparators map for it (not comparatorsForElementPropertyOrFieldTypes), lets call it comparatorsForTypes for now, not to affect field by field comparators. If the one to be introduced, should field by field comparators use both comparatorsForElementPropertyOrFieldTypes and comparatorsForTypes?
And should next pass:
assertThat(obiwan).extracting("name", "height")
  .usingComparatorForType(new BigDecimalComparator(), BigDecimal.class)
  .usingFieldByFieldElementComparator()
  .containsExactly("Obi-Wan", new BigDecimal("1.82"));

when next is not passing (as of now):
assertThat(obiwan).extracting("name", "height")
  .usingComparatorForElementFieldsWithType(new BigDecimalComparator(), BigDecimal.class)
  .usingFieldByFieldElementComparator()
  .containsExactly("Obi-Wan", new BigDecimal("1.82"));

?

@fiery-phoenix thanks for your comment, I agree with @PascalSchumacher to keep usingComparatorForElementFieldsWithType to make it clear it is only applied for element fields not elements themselves. Following that logic, I think that usingComparatorForType and usingComparatorForElementFieldsWithType should not share comparators.
Having said that, I feel adding usingComparatorForType is gonna confuse users, if the the difference between usingComparatorForElementFieldsWithType and usingComparatorForType might be understood, users might not get why a comparator registered with usingComparatorForType is not used on elements fields with the matching type, for example the following will not work:
public class Person {

    private final String name;

    public Person(String name) {
        this.name = name;
    }
}

@Test
public void test() {
    assertThat(new Person[] {new Person("John")})
        .usingComparatorForType(new AlwaysEqualStringComparator(), String.class)
        .usingFieldByFieldElementComparator()
        .containsExactly(new Person("Jane"));
}

do you think it  is easy to understand why the test fail ?

I think it will not be clear why the test you mentioned should fail. But now I have an idea what implementation could be less confusing. When have time, will update my pull request. Thank you.


Sec updates

Hey team,
This PR contains:

Fixes for 6 critical and high SSRF/XSS vulns (check Security tab for code scanning with CodeQL)
Fix for bigint not binding as part of the install, causing unexpected behaviour (e.g. https://github.com/solana-labs/governance-ui/actions/runs/6998523838/job/19036665741#step:6:120)
Deny listing script execution for 2 new packages
House keeping of github actions

@SebastianBor

Thanks


Added installation and usage sections to README

Just added a little documentation to the README. Thought it might be nicer than looking through the code. :)

This is great, thanks @tylermarien :smile:


Fix incomplete backup handling

Allow configuring if an incomplete backup should be a success or not.
Adds SUCCESS_ON_INCOMPLETE_BACKUP, defaulting to false.
Should fix #133

Does this need a review?

Yes and no. I tried to use this but it did not work as expected. It signaled incomplete backups as failed IIRC..
This kind of stuff is hard to test. Would be cool if someone else could test it as well.

Finally did enough testing to be confident that this works. Merging..


slyd: unrecognized service

Hi! I'm new.

This is a new installation
I can't reach the localhost:9001 from my host (it's all on a computer).
I can't start slyd

Config
Host
uname - a
Linux bigpc 4.8.0-37-generic #39-Ubuntu SMP Thu Jan 26 02:27:07 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
vagrant --version
Vagrant 1.9.1
Vagrantfile
# vim:ft=ruby

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.host_name = "portia"
  config.vm.provision :shell, :path => 'provision.sh', :args => ["install_deps", "install_splash", "install_python_deps", "configure_nginx", "configure_initctl"]
  config.vm.network "private_network", ip: "<IP_ADDRESS>"
  config.vm.network "forwarded_port", guest: 9001, host: 9001
  config.vm.provider "virtualbox" do |v|
    v.memory = 2048
    v.cpus = 1
  end
end

portia git:(master) cb-in ./VERSION
16.02.1
Guest
/etc/sudores
...
# User privilege specification
root    ALL=(ALL:ALL) ALL
vagrant ALL=(ALL) NOPASSWD: ALL
...

VBoxManage showvminfo portia_default_1486139928704_33506
https://gist.github.com/anonymous/34a66717a9422d66274c6221492845a2
vagrant@portia:~$ ps waux | grep 'slyd\|nginx'
root      1074  0.0  0.0  31556   800 ?        Ss   18:26   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     1076  0.0  0.0  31940  1644 ?        S    18:26   0:00 nginx: worker process
vagrant   1454  0.0  0.0  10460   936 pts/0    S+   18:33   0:00 grep --color=auto slyd\|nginx

Description
vagrant@portia:~# sudo service slyd restart
slyd: unrecognized service

Running into a similar issue on OS X (10.11.6).  I CAN start the slyd server but I can't connect:
~/d/portia ❯❯❯ telnet <IP_ADDRESS> 9001
Trying <IP_ADDRESS>...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.

I've tried restarting nginx and the slyd server but I still can't connect to the server.

As a follow up to my previous comment, I'm kind of amazed that any of this might work?  nginx.conf references files that need to be symlinked into /etc/nginx,  there are references to a dist folder but that seems to have been removed back in October.  Any ideas what's going on here?

@justnoise, what does sudo service slyd restart return?

@oshliaer Can you try running it using the new instructions and scripts here: https://github.com/scrapinghub/portia/pull/700

@ruairif Not success. It looks like #699 #701

The correct install instructions can be found here

@ruairif Thank you! Your job is great!


Prevent touches from scrolling if touchMove=false

This is implemented in pretty much the same way that draggable = false is.
I tested this on an app I'm writing that depends on this code. Let me know if there is a better way to test.

To disable touch swipe, you can set swipe props as false.
swipe = false feature is already working with react-slick


WIP config/v1: add PodSecurity feature gate to TechPreviewNoUpgrade

This enables PodSecurity admission (https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/2579-psp-replacement) if TechPreviewNoUpgrade feature set is enabled.
This depends on admission configuration being merged first: https://github.com/openshift/cluster-kube-apiserver-operator/pull/1217
This can be merged once 4.10 master opens, the earliest.
/hold

/lgtm

/close
(we did this elsewhere)


Why redux-form-website-template is listed as dependency in package.json

Question:
Why redux-form-website-template is listed as dependency in package.json

Oops. That was a mistake and an oversight. Thank you for pointing it out!


Refactor API Test

Is your feature request related to a problem? Please describe.
Current API test is hard to understand and maintain.
Describe the solution you'd like
Change to another framework for API test, which is better to maintain

Done.

Change to RestAssured
Migrate all existing test cases
Include API tests in git pre-push hook
Update CI script to run new tests
Remove old scripts, and test cases since no longer required.


[ARITH]const_int_bound doesn't allow INT64_MAX in expr

Patch to reproduce:
diff --git a/tests/python/unittest/test_arith_const_int_bound.py b/tests/python/unittest/test_arith_const_int_bound.py
index 84fc7fd64..0bcdff487 100644
--- a/tests/python/unittest/test_arith_const_int_bound.py
+++ b/tests/python/unittest/test_arith_const_int_bound.py
@@ -50,6 +50,15 @@ def test_cast_bound():
     assert bd.max_value == 2

+def test_bound():
+    import numpy as np
+    analyzer = tvm.arith.Analyzer()
+    x, y = te.var("x", "int64"), tvm.tir.const(-np.iinfo(np.int64).max, dtype="int64")
+    bd = analyzer.const_int_bound(x + y)
+    assert bd.min_value == bd.NEG_INF
+    assert bd.max_value == bd.POS_INF
+
+
 def test_add_sub_bound():
     analyzer = tvm.arith.Analyzer()
     x, y = te.var("x", "int64"), te.var("y", "int64")

Because the expr is  (x + (int64)-9223372036854775807), it runs into this ICHECK
E             File "/home/xiaoquan.li/tvm/src/arith/const_int_bound.cc", line 455
E           TVMError:
E           ---------------------------------------------------------------
E           An internal invariant was violated during the execution of TVM.
E           Please read TVM's error reporting guidelines.
E           More details can be found here: https://discuss.tvm.ai/t/error-reporting/7793.
E           ---------------------------------------------------------------
E             Check failed: y != kNegInf == false:

Since -INT64_MAX is used as kNegInf in const_int_bound, does it mean this value shouldn't be used in TIR?

maybe related https://github.com/apache/incubator-tvm/pull/6911

Please check if #6911 resolves the issue

Thanks all, it is solved.


Make CMake on TensorFlow Incrementally Compile on Windows

I'm working with the tensorflow r1.4 branch on Windows 7 with Visual Studio 2015.   When I configure
cmake C:\Users\Kevin\dev\tensorflow-r1.4\tensorflow\contrib\cmake -A x64 -DCMAKE_BUILD_TYPE=Release -DSWIG_EXECUTABLE=C:/Users/Kevin/dev/swigwin-3.0.12/swig.exe -DPYTHON_EXECUTABLE=C:/ProgramData/Anaconda3/python.exe -DPYTHON_LIBRARIES=C:/ProgramData/Anaconda3/libs/python35.lib -DPYTHON_INCLUDE_DIR=C:\ProgramData\Anaconda3\include -DNUMPY_INCLUDE_DIR=C:\ProgramData\Anaconda3\Lib\site-packages\numpy\core\include -Dtensorflow_ENABLE_GPU=ON -DCUDNN_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0"

and run
powershell "MSBuild /m /p:Configuration=Release tf_python_build_pip_package.vcxproj | tee msbuild.txt"

the build always compiles all the projects, even the projects already compiled.
Looking at the first few lines of msbuild.txt I see
Microsoft (R) Build Engine version 14.0.25420.1
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/2/2017 4:35:58 PM.
     1>Project "C:\users\kevin\dev\tensorflow-r1.4\tensorflow\contrib\cmake\tf_python_build_pip_package.vcxproj" on node 1 (default targets).
     1>Project "C:\users\kevin\dev\tensorflow-r1.4\tensorflow\contrib\cmake\tf_python_build_pip_package.vcxproj" (1) is building "C:\Users\Kevin\dev\tensorflow-r1.4\tensorflow\contrib\cmake\ZERO_CHECK.vcxproj" (2) on node 1 (default targets).
     2>InitializeBuildStatus:
         Creating "x64\Release\ZERO_CHECK\ZERO_CHECK.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
       CustomBuild:
         Checking Build System
         CMake is re-running because C:/Users/Kevin/dev/tensorflow-r1.4/tensorflow/contrib/cmake/CMakeFiles/generate.stamp is out-of-date.
           the file 'C:/Users/Kevin/dev/tensorflow-r1.4/tensorflow/contrib/cmake/CMakeFiles/tf_core_gpu_kernels.dir/__/__/core/kernels/tf_core_gpu_kernels_generated_cwise_op_gpu_igammas.cu.cc.obj.depend'
           is newer than 'C:/Users/Kevin/dev/tensorflow-r1.4/tensorflow/contrib/cmake/CMakeFiles/generate.stamp.depend'
           result='-1'
         -- Configuring done

The issue isn't the multicore builds, I'm just using multicore builds to speed up the overall build, which takes 5 hours on one core.
The problem appears to be that the generate.stamp isn't what's expected.

This would definitely be a useful feature for anyone relying on the CMake build in regular development. However, since we only currently rely on CMake for CI builds on Windows, these are always from scratch, and we hope to shift that build to Bazel for Windows ASAP, it is unlikely that anyone on the TensorFlow team will spend time investigating the cause of the gratuitous rebuilds. Therefore I'm going to mark this issue as "Contributions Welcome", and I hope some member of the community heeds the call.
I'd be happy to respond to questions on this thread about how the build is structured with a view to making it incremental, and very happy to review pull requests that improve the situation.
/cc @gunan FYI.

When will the Windows Bazel build be ready for prime-time?
The Nettles: Celtic. Eclectic. Folkadelic.
www.TheNettles.com http://www.thenettles.com/
On Thu, Nov 2, 2017 at 5:44 PM, Derek Murray<EMAIL_ADDRESS>wrote:

This would definitely be a useful feature for anyone relying on the CMake
build in regular development. However, since we only currently rely on
CMake for CI builds on Windows, these are always from scratch, and we hope
to shift that build to Bazel for Windows ASAP, it is unlikely that anyone
on the TensorFlow team will spend time investigating the cause of the
gratuitous rebuilds. Therefore I'm going to mark this issue as
"Contributions Welcome", and I hope some member of the community heeds the
call.
I'd be happy to respond to questions on this thread about how the build is
structured with a view to making it incremental, and very happy to review
pull requests that improve the situation.
/cc @gunan https://github.com/gunan FYI.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/14194#issuecomment-341598311,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AOw_oWEvDPRQZanxxGw6gPX3_jk_X8hVks5symHTgaJpZM4QQfT-
.

Hi @johnsrude , now it lacks tf.load_library() support. All stuffs in tf.contrib are missing.   For the others, it's almost perfect.

@johsrude The CMake build appears to be configured to generate an new version file tensorflow/core/util/version_info.cc for every build, which is included in the low level tf_core_framework.  This is at least partly responsible for the rebuild problems.
There is a custom command that creates the version file from python + git here:
https://github.com/tensorflow/tensorflow/blob/dcd820616d352d1e8844b1db504af4e859176cbf/tensorflow/contrib/cmake/tf_core_framework.cmake#L271-L283
I think what we want, is to only update this file if the git version has changed.  I've added an extra add_custom_command to copy the output from gen_git_source.py --raw_generate overtensorflow/core/util/version_info.cc only if it has changed using CMake's copy_if_different, and that seems to address the unwanted full rebuild.
# Tricky setup to force always rebuilding
# force_rebuild always runs forcing ${VERSION_INFO_CC} target to run
# ${VERSION_INFO_CC} would cache, but it depends on a phony never produced
# target.
set(gen_git_source ${tensorflow_source_dir}/tensorflow/tools/git/gen_git_source.py)
set(version_info_cc_tmp ${CMAKE_CURRENT_BINARY_DIR}/version_info.cc)
add_custom_target(force_rebuild_target ALL DEPENDS ${VERSION_INFO_CC})
set_property(TARGET force_rebuild_target PROPERTY FOLDER "custom")

add_custom_command(OUTPUT __force_rebuild COMMAND ${CMAKE_COMMAND} -E echo)
add_custom_command(OUTPUT
    ${version_info_cc_tmp}
    COMMAND ${PYTHON_EXECUTABLE} ${gen_git_source} --raw_generate ${version_info_cc_tmp}
    DEPENDS __force_rebuild)

set(VERSION_INFO_CC ${tensorflow_source_dir}/tensorflow/core/util/version_info.cc)
add_custom_command(OUTPUT
  "${VERSION_INFO_CC}"
  DEPENDS "${version_info_cc_tmp}"
  COMMAND ${CMAKE_COMMAND} -E copy_if_different "${version_info_cc_tmp}" "${VERSION_INFO_CC}"
  )

set(tf_version_srcs ${VERSION_INFO_CC})

I've tested this on OS X and it seems to be working.  Is there an active CMake author who can review this?

@headupinclouds I'd be happy to review a PR for this. Thanks!

I've pushed the above changes in a PR (tested in another branch) for now:
https://github.com/headupinclouds/tensorflow/tree/pr.cmake.incremental.build
I'm running into some unrelated build issues after pulling the latest from master.  I'll hold off on submitting this until I sort those out.

This seems very useful. Every time I compile a user_op on windows it needs to re-compile everything..

Hi @steven-hh-ding Forget cmake, Embrace bazel!

@snnn Thanks! You mean I can build user_opr on windows with bazel? I will check it out!

Wait, for custom user_op, if you build it as a dll, then you can put it into a separated project, which is not part of the tensorflow.sln. So you don't have to use either cmake or bazel. All the issues described here are gone.
If you put your custom user_op inside tensorflow source tree, it's another story. Yes. you can build it with bazel, and it's very fast.

Thank you @snnn ! I managed to skip the rebuild with /p:BuildProjectReferences=false. Will switch to bazel later. Thanks!

with up-to-date sources (--source_dir and --git_tag_override):
# Tricky setup to force always rebuilding
# force_rebuild always runs forcing ${VERSION_INFO_CC} target to run
# ${VERSION_INFO_CC} would cache, but it depends on a phony never produced
# target.
set(gen_git_source ${tensorflow_source_dir}/tensorflow/tools/git/gen_git_source.py)
set(version_info_cc_tmp ${CMAKE_CURRENT_BINARY_DIR}/version_info.cc)
add_custom_target(force_rebuild_target ALL DEPENDS ${VERSION_INFO_CC})
set_property(TARGET force_rebuild_target PROPERTY FOLDER "custom")

add_custom_command(OUTPUT __force_rebuild COMMAND ${CMAKE_COMMAND} -E echo)
add_custom_command(OUTPUT
    ${version_info_cc_tmp}
    COMMAND ${PYTHON_EXECUTABLE} ${gen_git_source}
    ARGS --raw_generate ${version_info_cc_tmp} --source_dir ${tensorflow_source_dir} --git_tag_override=${GIT_TAG_OVERRIDE}
    DEPENDS __force_rebuild)

set(VERSION_INFO_CC ${tensorflow_source_dir}/tensorflow/core/util/version_info.cc)
add_custom_command(OUTPUT
  "${VERSION_INFO_CC}"
  COMMAND ${CMAKE_COMMAND}
  ARGS -E copy_if_different "${version_info_cc_tmp}" "${VERSION_INFO_CC}"
  DEPENDS "${version_info_cc_tmp}"
  )

set(tf_version_srcs ${VERSION_INFO_CC})

I would avoid to build tensorflow at home because my internet service provider(Xfinity) has a data cap.

@johnsrude It seems you are using older versions(1.x versions) of Tensorflow which is not supported any more. Contrib has been depreciated in Tensorflow 2.x , also please refer this link .


[Fix-10909] Returns executionStatus for each process instance status.

Purpose of the pull request

Returns executionStatus for each process instance status.
Add accessToken resource module.
some checkstyle fix.

Brief change log
close #10909
Verify this pull request
Manually verified the change by testing locally.

replace by #10922


Replace summarize_mode with commitable_code_suggestions_mode in p…

User description
…r_code_suggestions_prompts.toml to enhance PR suggestion mechanism

Type
enhancement

Description

Replaced summarize_mode with commitable_code_suggestions_mode in the configuration file to improve the PR suggestion mechanism.

Changes walkthrough
Relevant filesEnhancement

pr_code_suggestions_prompts.tomlEnhance PR Suggestion Mechanism in Configuration

pr_agent/settings/pr_code_suggestions_prompts.toml
Replaced summarize_mode with commitable_code_suggestions_mode to enhance PR suggestion mechanism.

+2/-2

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

/review auto_approve

/improve


Question: how to send welcome message for new user in channel by using tdlib

i find bot cant do that after read telegram-bot document.
search in google but cant find useful info. I want to know how tdlib can get the event when a new user joins
thank you for your answer

Only bots can get an event when a user joins a channel. If the user sent a join request instead, then the bot can contact the user. Otherwise, there is no way to send a welcome message to the user. You can just pin it in the channel instead.


Upgrade Zig Version to 0.13.0

Is there anything stopping us from upgrading to the latest Zig version (0.13.0)?
Current Zig version:
https://github.com/roc-lang/roc/blob/1bcf30391baef2cd19ff2158ba78748b67d3ddd1/nix/compile-deps.nix#L3
I am encountering issues building from source using Nix on Darwin aarch64:
https://github.com/NixOS/nixpkgs/issues/299091

We discussed it briefly at https://roc.zulipchat.com/#narrow/stream/395097-compiler-development/topic/Zig.200.2E12.2E0/near/444208611
Essentially just needs someone to make it happen I think.

great to see it happening in #6921 thank you @lukewilliamboswell

@skyqrose, can this be closed now that #6921 is merged?


Support for transparent compositional relationships

With the current API, it's very unwieldy to create a composite model, where information is loaded into the model as flat attributes instead of loadable relations.
The use case I'm currently looking at is something like this: relevant tables are entity, entity_revision, and entity_data. An entity can have many revisions and has a single master revision, and each revision has an entity_data entry. I want to present a single Entity model which incorporates columns from both entity and entity_data. Entities are typically fetched by ID, but in some circumstances it would also be useful to fetch a specific revision of a given entity.
Currently, handling this use case requires either manually creating virtuals for every composited field (which removes the benefit of using reflection) or by creating a view and triggers in the database to do the heavy lifting. Instead, the current tableName functionality could be extended to allow definition of a composite model based on relationships between the tables. This may, of course, be a project for bookshelf-next.

Yes, it's a great idea. It would be quite a big task.

The project leadership of Bookshelf recently changed. In an effort to advance the project we close all issues older than one year.
If you think this issue needs to be re-evaluated please post a comment on why this is still important and we will re-open it.
We also started an open discussion about the future of Bookshelf.js here https://github.com/bookshelf/bookshelf/issues/1600. Feel free to drop by and give us your opinion.
Let's make Bookshelf great again

I already would be happy to get related data as flat object. Right now, by invoking .toJSON I get the related data as nested objects. The API consumer is not interested or concerned with how especially 1:1 relations are handled, but would rather have the data extended on the main object. Is that possible yet with bookshelf?

Nope. You're free to work on it if you'd like.


Toggl Track Button disappeared from Trello

Windows 11:
Chrome recent:
recent:

Relevant integration (if any):
trello.com
🐛 Describe the bug
The Toggl Button disappeared from Trello when you open a card.
Expected behaviour
When I open a card in Trello, there is the Toggl Track button on the right hand side
Steps to reproduce
use toggl, activate integration with trello, open trello, see no button
Other details or context
It stopped working on 2023-12-13

It has disappeared for me as well.

Case reference

Case reference

Case reference

Same here:

Windows 10 build 19045
Chrome v120.0.6099.130 (latest stable)

For clarity's sake:
Trello being https://trello.com/
and integration for the website enabled:

I've worked around similar situations by reloading the Trello page with browser cache cleared, but this time the problem doesn't solve itself.

As a follow-up to my last comment above, the issue has resolved for me.

I faced the issue when I changed my PC.
My Chrome browser was synced, so the toggl extension was installed automatically in my new PC browser. The Integrations settings in the toggl extension (which users can access via chrome-extension://<extension-id>/settings.html) were also reflected on my new browser. The checkbox for trello.com was enabled.
However, *://*.trello.com/* was not among the list of site access permissions for the toggl extension on my new browser (which users can access via chrome://extensions/?id=<extension-id>, which was probably the reason why I didn't see the toggl track button on trello.com web pages.

When I un-checked the Integrations checkbox for trello.com and re-checked it, the dialog for Chrome asking me whether to permit toggl extensions access for trello.com appeared. Consenting to this has solved the issue for me.

Still seeing this issue today. If I open a card and then refresh the page the button appears. But when I navigate within the app it no longer appears, so it probably has something to do with the detection of when a card is open and when the button should be added to the page.


03/07/2018 13:23 : Create Pull Request for branch prTesting_2018030713 -> master

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestCon.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


CBLAuthDelegate:receiveTokenResponseFailed:result=AUTHORIZATION_PENDING

IMPORTANT: Before you create an issue, please take a look at our Issue Reporting Guide.
Briefly summarize your issue:
I'am not recieving the device code and link to get my device authorised.
What is the expected behavior?
##################################
NOT YET AUTHORIZED
##################################
################################################################################################
To authorize, browse to: 'https://amazon.com/us/code' and enter the code: {XXXX}
################################################################################################
###########################
Authorized!
###########################
########################################
Alexa is currently idle!
########################################
What behavior are you observing?
##################################################
Checking for authorization (18)...
##################################################
2020-02-27 03:16:18.737 [  3] 5 HttpPost:doPostSucceeded:code=400
2020-02-27 03:16:18.739 [  3] 5 CBLAuthDelegate:receiveTokenResponse:code=400
2020-02-27 03:16:18.740 [  3] 5 CBLAuthDelegate:mapHTTPStatusToError:code=400,error=INVALID_REQUEST
2020-02-27 03:16:18.740 [  3] 5 CBLAuthDelegate:errorInLwaResponseBody:error=authorization_pending,errorCode=AUTHORIZATION_PENDING
2020-02-27 03:16:18.741 [  3] 5 CBLAuthDelegate:setAuthError:authError=AUTHORIZATION_PENDING
2020-02-27 03:16:18.742 [  3] 0 CBLAuthDelegate:receiveTokenResponseFailed:result=AUTHORIZATION_PENDING
2020-02-27 03:16:23.746 [  3] 5 CBLAuthDelegate:requestToken
Provide the steps to reproduce the issue, if applicable:
Double clicking startsample.bat.
Tell us what hardware you're using:

[* ] Desktop / Laptop
[ ] Raspberry Pi
[ ] Other - tell us more:

Tell us about your OS (Type & version):

[ ] Linux
[ ] MacOS
[ ] Raspbian Stretch
[ ] Raspbian Jessy
[* ] Other - Windows 10

How did you fix this issue? I am having the same thing only on a raspberry pi

I have the same problem. but he never authorizes me. I have tried deleting cblAuthDelegate.db and generating a new config.json. I understand that the config.json prevents the error of mistyping the clientID. I'm working on bullseye in raspberry.
can it compile without errors in bullseye?

Same here. It never authorizes and says Authorization pending. I redid the security profile, I activated it, checked the config and Alexa Integration files over and over again. I recompiled the SDK with and without DKCS11 as I saw somewhere that it is now required, but this changed nothing.
It goes into this loop of Checking for Authorization... :-(


limit cluster wide access to secrets

Giving to k8s-image-availability-exporter cluster wide access to all secrets is a little concern. As far as I understand it's to be able to use container registry credentials, right?
So it would be useful to have option, eg.:

Disable such a wide access to every secret in the cluster if we don't use any images from private registries.
Currently if we want limit access to secrets by removing from RBAC:

  - apiGroups:
      - ""
    resources:
      - secrets
    verbs:
      - list
      - watch
      - get

it's causing k8s-image-availability-exporter fail to start with:
E0529 18:35:41.985749       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20191016111102-bec269661e48/tools/cache/reflector.go:96: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:kube-system:image-availability-exporter" cannot list resource "secrets" in API group "" at the cluster scope

Limit access to secrets for k8s-image-availability-exporter to only particular namespaces.

Hi. Sorry for the delay.
Yes, the concert is valid. I'll see if I can address it this week.

The current approach is extremely simple and efficient since it creates a global API Watch, and cross-references pullSecretReferences with the secrets directly in the memory of the image-availability-exporter.

Hi team,
do you have any plans regarding the issue eventually?

For now the issue is in the help wanted status, because the task is not in our nearest backlog. However, we  appreciate any ideas and any help with the implementation.

One of my ideas was to pass the label selector for namespaces that the exporter can monitor. Then, filter namespaces by the selector and subscribe to secrets controllers and pods only in the limited scope of the cluster.

One of my ideas was to pass the label selector for namespaces that the exporter can monitor
Vote for this. We are currently thinking of using the service, but security concerns force us to either analyze any alternatives or develop extra controls, since:

the service account has access to all secrets in the cluster, meaning anyone with proper RBAC permissions (e.g. impersonation to the service account) can gain these privileges which is critical for shared environments;
the pod stores information about all cluster secrets in memory, meaning it’s critical to restrict any kind of ‘exec’ access to the pod; but since the image is built on top of distroless, it looks as a minor concern.

There is already a label to select namespaces, oh my. Thus, the only thing required is to refactor the informers to look into specific namespaces instead of requesting resources clusterwide. Then, clusterwide secret access will not be required anymore, it will be enough to create roles/rolebindings in desired namespaces (if needed).


fasterrcnn does not work with forward hooks (including wandb)

It appears that faster-rcnn does not work properly with fastai (2.1.4) and WandB. This appears to be a bug in WandB.
I have a model that trains without WandB. When I add the WandB callback, it crashes. Identifying the precise point is not easy. In the same environment, efficientdet works with fastai and the WandB callback.
Traceback (most recent call last):
  File "/home/adam/anaconda3/envs/dlm/lib/python3.8/site-packages/fastai/learner.py", line 155, in _with_events
    try:       self(f'before_{event_type}')       ;f()
  File "/home/adam/anaconda3/envs/dlm/lib/python3.8/site-packages/fastai/learner.py", line 164, in _do_one_batch
    self.pred = self.model(*self.xb)
  File "/home/adam/anaconda3/envs/dlm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/adam/anaconda3/envs/dlm/lib/python3.8/site-packages/torchvision/models/detection/generalized_rcnn.py", line 80, in forward
    images, targets = self.transform(images, targets)
  File "/home/adam/anaconda3/envs/dlm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 740, in _call_impl
    var = var[0]
TypeError: 'ImageList' object is not subscriptable

This error occurs in wandb_torch.py in nested_shape (line 20). It iterates for item in list(array_or_tuple) but the value of array_or_tuple is a torchvision.models.detection.image_list.ImageList object. This has two attributes: image_sizes (for a batch size of 8, this could be a list of tuples (512, 512) *8; and tensors, which is actually a single tensor, this could be a tensor of shape (8,3,512,512). So the imagelist is not sub-scriptable and the test fails.

bug in wandb_torch / pytorch / detection
The problem occurs in pytorch module.py and is triggered by the WandB forward hook.

If wandb is not added, then there is no forward hook and the problem in module.py is not triggered

If wandb is added, then there is a forward hook. The output of the fasterrcnn module is a tuple (ImageList, dict). The code in
module.py expects to be able to keep going down the left-most piece of output until it finds a tensor (see var=var[0] below). But this doesn't work with the ImageList object. Hence the 'ImageList' object is not subscriptable exception.

I'm not really sure if the bug here is in pytorch module, pytorch detection, or wandb. I think that ImageList is probably not respecting the contract (implicit) of the pytorch module code.

I previously thought that the problem was with wandb nested_shape, which basically makes the same assumption as pytorch module. In fact, the output of nested_shape does not appear to be used directly in the code other than being stashed in a node object under the output_shape attribute.

          for hook in itertools.chain(
                _global_forward_hooks.values(),
                self._forward_hooks.values()):
            hook_result = hook(self, input, result)
            if hook_result is not None:
                result = hook_result
        if (len(self._backward_hooks) > 0) or (len(_global_backward_hooks) > 0):
            var = result
            while not isinstance(var, torch.Tensor):
                if isinstance(var, dict):
                    var = next((v for v in var.values() if isinstance(v, torch.Tensor)))
                else:
                    var = var[0]

To use WandB with FasterRCNN, you can call wandb.init with log_parameters=False and log_gradients=False . These two loggers are implemented with a forward hook that triggers the bug above.

@FraPochetti didn't we faced this issue (or a very similar one) before?

Thanks for pointing out that this is basically a duplicate of #527 and #490
One minor improvement to the current patch would be to catch the wandbcallback in calls to instantiate Model and Learn.

One minor improvement to the current patch would be to catch the wandbcallback in calls to instantiate Model and Learn.

We're currently catching it on the Learn instantiation, what you have in mind for catching while instantiating the model?

Won't fix.


Install a 3rd-party plugin for PowerToys Run

Description of the new feature / enhancement
First, I would like to say thank you guys for the great tools for the Windows user. I am wondering if the user can install a 3rd-party plugin like the command wpm install in WOX or if we can install the plugin in the settings UI. The tool will be more powerful if we can support a 3rd-party plugin.
Scenario when this would be used?
Installing more plugin for PowerToys Run.
Supporting information
No response

there are 3rd party plugins for PTR (not that many tho), but they have to be compiled for PTR, you can't just take them from WOX as is.
example of 3rd party plugin here, it's not included in the official build because it relies on 3rd party executables and libraries to run, which the devs here don't want to include for security reasons.

/dup #3200


Integrate algorithms & data format

The task is to adjust algorithms' interface to use common data format from #34.

 Integrate .NET interfaces & dataset

Closed as 2e88b5a


Add ts18046 to known errors

https://typescript.tv/errors/#ts18046

@sindresorhus can you release this?

https://github.com/tsdjs/tsd/releases/tag/v0.30.7


Normalise Austlang codes

Issue contributed by Stacey van Groll<EMAIL_ADDRESS>When there are Austlang languages, parse the data to only the code to map against the Customised Languages mapping table.
Austlang languages are referred to with first the language which may be one, two, or three words with characters between them, then a space, then the word language, then a space, and then the code for the language.
There are 1,200+ Austlang codes. The code always has an upper case character first, then may have one, two or three digits, then may have a full stop, and then the full stop may be followed by a digit or an upper case character. The Austlang data needs to be in an exact form in order to be mapped against the Customised Languages mapping table, with entries such as lang.a89 The record must also include dc:language field of aus but may also have other data such as an additional entry of eng.
The outcome is correct representation in UI by Language facet and Language Details record display, with facet values of both Australian languages and the Austlang language, allowing for nested or granular language selection. In Primo Back Office, this is handled by a normalisation rule parsing the data to obtain the code and transform to necessary form by the most structured aspect, being the code at the end of the string, which is always preceded by the word language, then a space, and then the code.
For example,

By coincidence, ExLibris updated the “Customized Languages Codes” table, adding the 1500+ Austlang codes in the May 5th release. It may have affected the code naming taxonomy in the main “Language codes Labels” table so I will look at getting the code updated to align with the new code names.

Final code committed and issue closed

I find that the data is not normalised from the code to the language in the full record display when a lower case prepend of lang is included, despite that the Customized Language code table entries include lang as a prepend. For example dc:language of lang.e81 displays in the full record Language field as e81 when it should display as Jarowair. I have confirmed that the full record display Language field is as expected when adjusting to remove this portion: add prefix (TEMP"1","lang.")
I’m not sure if it’s some sort of indexing issue, or something else I’m missing in the configuration, but currently the data is only being added to display and not the search (and therefore not also the facet)
The rule only works on one instance of an Austlang code language per record, when it should take effect on all instances. For example, a record with Jarowair language E81 and Wakka Wakka language E28 only displays Jarowair and does not display Wakka Wakka. I am having a great deal of trouble trying to figure out a solution for this given the apparent limitations of Xpath version 1.0 which does not allow looping or conditions within the rule, and still avoids duplicated data
Additional note: The comment on April 23 2024 is that Ex Libris updated the Austlang codes on May 5th Release for Primo VE. I’m not sure where this information came from, as the codes have been in place from October 2021 for Primo VE, I can find no record of this nor see any changes. But I don't think this is related to the other issues I've mentioned

Kia ora Stacey,
Thank you for your comments.

It is interesting that only the language code is required and not the "lang." prefix. I'll modify the code. Thank you.
Adding search and facet data is a two step process. You would first import and normalise the data and ensure the is in the desired dc field. Step two is accomplished by adding local fields. Once you have created the local field, you can enable it for search and/or faceting. You then configure the "DC Normalise Rules" panel to add a drool for the appropriate value in the display, index and facet sections.
Yes this is the real limitation and is a known issue. If you have multiple values, then you will need multiple instances of the rule.
Agreed, this shouldn't affect the normalisation.

Let me know if you have any other questions.
Ngā mihi,
Craig.

Hi Craig,

This is not mapping to a local field, but is utilising the OTB Language field. Apologies if I'm missing something here, but I have adjusted other XML external data sources by normalisation rules for similar local fields like Author and Subject, and the data appears in the search>facet for Primo VE without any extra effort. I'm suspecting this is a defect.

I am finding this difficult to create a rule with multiple iterations for a potentially very high number of languages in Austlang that may be in our records which also does not result in unexpected data, including by apparent inability in XPath 1.0 to check for the field already existing and handle appropriately. For example:
Primo BO data: aus;eng;C15;C6;C8;C7.1;C8.1
Primo VE data: "language" : [ "aus;eng;c15;c6;c8;luritja language c7.1;c8.1;pitjantjatjara language;eastern arrernte language;western arrernte language;anmatyerre language;ukatja language" ],

New items:
5. I have found another defect with VE that it fails to display "Undetermined" if the text in the Language field is not present in the relevant Language mapping table.
Instead it displays the text normalised by the rule, which worsens the problems noted above in 3 and can be seen in the example above.

I find that the rule currently is not strict enough, in being based on the presence of the word 'language' in the subject, as there are other non-Austlang subjects with the word 'language'.
For example:
Raw subject data: English language -- Dialects -- Queensland
“Language” display field: dialects queensland
The regular expression I created in BO accounts for the specs of complexities I determined for Austlang, with Austlang languages having first the language which may be one, two, or three words with characters between them, then a space, then the word language, then a space, and then the code for the language. The code always has an upper case character first, then may have one, two or three digits, then may have a full stop, and then the full stop may be followed by a digit or an upper case character.
Given these complexities, I determined it was better to work backwards from the end of the string to separate out just the necessary code to reference against the mapping table, and implemented a BO transformation to take all matching strings by: language\s([A-Z]\d{1,3}(.[A-Z0-9]+)?)$

I have opened a case regarding these problems, which is high priority also as I have received multiple assurances from Ex Libris Primo Product Management and via the Go VE channel that this functionality is supported in Primo VE, as a feature alignment issue from Primo BO. This includes from the documentation published in 2021, and correspondence in December 2023 and April 2024.
Cheers,
SvG

Hi Stacey,
I've been out of the office for most of the last two days (office party season) so I'll have a look at your comments and get back to you tomorrow.
Cheers,
Craig.
From: SvanGroll @.>
Sent: Wednesday, 11 December 2024 11:40 am
To: primousers/primo-ve-norm @.>
Cc: Digital Access @.>; State change @.>
Subject: Re: [primousers/primo-ve-norm] Normalise Austlang codes (Issue #5)
Caution: This email originated from outside our organisation. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi Craig,

This is not mapping to a local field, but is utilising the OTB Language field. Apologies if I'm missing something here, but I have adjusted other XML external data sources by normalisation rules for similar local fields like Author and Subject, and the data appears in the search>facet for Primo VE without any extra effort. I'm suspecting this is a defect.
I am finding this difficult to create a rule with multiple iterations for a potentially very high number of languages in Austlang that may be in our records which also does not result in unexpected data, including by apparent inability in XPath 1.0 to check for the field already existing and handle appropriately. For example:
Primo BO data: aus;eng;C15;C6;C8;C7.1;C8.1
Primo VE data: "language" : [ "aus;eng;c15;c6;c8;luritja language c7.1;c8.1;pitjantjatjara language;eastern arrernte language;western arrernte language;anmatyerre language;ukatja language" ],

New items:
5. I have found another defect with VE that it fails to display "Undetermined" if the text in the Language field is not present in the relevant Language mapping table.
Instead it displays the text normalised by the rule, which worsens the problems noted above in 3 and can be seen in the example above.

I find that the rule currently is not strict enough, in being based on the presence of the word 'language' in the subject, as there are other non-Austlang subjects with the word 'language'.
For example:
Raw subject data: English language -- Dialects -- Queensland
"Language" display field: dialects queensland
The regular expression I created in BO accounts for the specs of complexities I determined for Austlang, with Austlang languages having first the language which may be one, two, or three words with characters between them, then a space, then the word language, then a space, and then the code for the language. The code always has an upper case character first, then may have one, two or three digits, then may have a full stop, and then the full stop may be followed by a digit or an upper case character.
Given these complexities, I determined it was better to work backwards from the end of the string to separate out just the necessary code to reference against the mapping table, and implemented a BO transformation to take all matching strings by: language\s([A-Z]\d{1,3}(.[A-Z0-9]+)?)$

I have opened a case regarding these problems, which is high priority also as I have received multiple assurances from Ex Libris Primo Product Management and via the Go VE channel that this functionality is supported in Primo VE, as a feature alignment issue from Primo BO. This includes from the documentation published in 2021, and correspondence in December 2023 and April 2024.
Cheers,
SvG

Reply to this email directly, view it on GitHubhttps://github.com/primousers/primo-ve-norm/issues/5#issuecomment-2533102583, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABL4QMFQLTAMV2SEQ7VBHUT2E5UTDAVCNFSM6AAAAABTF7FNMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZTGEYDENJYGM.
You are receiving this because you modified the open/close state.Message ID<EMAIL_ADDRESS>
"The contents of this e-mail (including any attachments) may be confidential and/or subject to copyright. Any unauthorised use, distribution, or copying of the contents is expressly prohibited. If you have received this e-mail in error, please advise the sender by return e-mail or telephone and then delete this e-mail together with all attachments from your system."


Cannot upload with external dependency due to "Invalid value for requires_dist"

What's the problem this feature will solve?
I cannot upload a fully functioning wheel to pypi:
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Invalid value for requires_dist. Error: Can't have direct dependency: 'dictdiffer @ git+git://github.com/mafrosis/dictdiffer@merge-ignore'

Describe the solution you'd like
Allow packages to be uploaded when they reference an externally hosted wheel dependency.
Additional context
Apologies if this is a repost of an old issue (such as #7136).
My use-case is when a dependency of my package has a bug which I need to fix, but they are unable to release the patch to pypi. This effectively blocks new releases of my package.
A simple solution is to temporarily depend on a binary distributed on a different platform to pypi (ie, Github).
In this specific instance, I packaged a wheel on my Github fork. I update my dependencies to point at this new binary target (instead of a just pointing at a branch on a source repo).
At this point, everything works - distutils is happy and pip can install. I just can't share the work with anyone via pypi.

Thanks for filing the issue, but this is indeed a duplicate of #7136 and we don't have any plans to change the behavior here.
If the project in question is having issues releasing to PyPI, feel free to have them file an issue and we can attempt to help.

Thanks for coming back to me quickly with an answer.
For future readers, I'd like to note that I do think this is a real issue. Playing devil's advocate for a moment:
Imagine if my downstream dependency has a critical security flaw which needs to be patched immediately, but their CI is broken and their maintainer is on holiday? Where does that leave all the packages that need to ship a patch?
To be clear, I'm not saying someone need's to fix this - but I do think this bug restricts a certain class of very important fixes being released to the community.

Consider if you were able to do that: that release would be forever tied to your direct dependency, you would need to maintain  the git repo indefinitely, it wouldn't be able to go away or that version would become broken, and it also wouldn't benefit once the affected/forked project made a proper new release, as it would continue using the direct dependency.
While in this specific case the existing behavior would mean a potentially slower "fix" for your project, in the long term it results in a much less broken, fractured & forked ecosystem.

I understand, and respect that it's not my call :)
To answer your question - I expect to "yank" that release once the fix makes it upstream, and so for some short indeterminate amount of time, the "hacked" release would continue to work.. But likely would break in the future. It should be up to my project to determine how short that "short" time is for our ecosystem and users.

You can fork the project on PyPI. You can push your own version of the package as a new package that you control, ideally with a long and explicit name and a very clear note in the short + long description, and have your downstream package reference your fork instead of the real one. It's the equivalent of your git solution but without git.

That's absolutely right. I'm not sure that this is a better result than simply allowing packages to depend on binaries hosted by Github.
https://pypi.org/project/dictdiffer-jira-offline-fork/


Switch csv parsing to csv library from binary-csv

binary-csv has less support for options and file encodings (see #1 for a related bug). Propose we switch to standard node csv library.

I recommend closing based on https://github.com/frictionlessdata/jsontableschema-infer-js/issues/3

WONTFIX. See #3 (deprecating this library).


Only render a Ratings element if there are ratings to display. r=vporof"

Coverage decreased (-0.01%) to 43.396% when pulling c59b44cebec3ad02ffc189132378ff321933f315 on jsantell:optional-ratings into 6c422b3b98ea63567e878d84a5dcd6c446fba38a on mozilla:master.


Documentation website

Is there going to be a documentation website in the future?
Or Is the Readme all there is to it?
I was asking as I saw a docs folder in the repository.

found it  at https://fastprogress.fast.ai/
Please do add it to the about section for more visibility


Add high risk application/service principal permissions into ScubaResults.json

🗣 Description

Adds a new module AADRiskyPermissionsHelper.psm1 which has the following functions:

Get-ApplicationsWithRiskyPermissions
Get-ServicePrincipalsWithRiskyPermissions
Get-FirstPartyRiskyApplications
Get-ThirdPartyRiskyServicePrincipals
Format-RiskyPermission
Format-Credentials
Merge-Credentials

These functions serve the purpose of parsing application and service principal objects for risky permissions. The result can be found in ScubaResults.json with the following keys: first_party_risky_applications and third_party_risky_service_principals.
First-party applications refer to applications created within a local tenant, while third-party service principals refer to service principals owned by an external tenant.
New Microsoft commandlets added:

Get-MgBetaApplication
Get-MgBetaApplicationFederatedIdentityCredential
Get-MgBetaServicePrincipal
Get-MgBetaServicePrincipalAppRoleAssignment

💭 Motivation and context

Closes #1327
Part of ongoing work for epic #1073
Created #1463 as a follow-up issue
🧪 Testing

How to run Pester tests
Checkout this PR's feature branch locally, then from the the root directory run the following commands:
Invoke-Pester -Output 'Detailed' -Path '.\PowerShell\ScubaGear\Testing\Unit\PowerShell\Providers\AADProvider\AADRiskyPermissionsHelper\'

Export-AADProvider.Tests.ps1 was modified since we call the above functions in Export-AADProvider.psm1. Run all AAD provider tests to ensure they pass:
Invoke-Pester -Output 'Detailed' -Path '.\PowerShell\ScubaGear\Testing\Unit\PowerShell\Providers\AADProvider\

Create a new report, verify first_party_risky_applications and third_party_risky_service_principals are added in ScubaResults.json.
✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] PR targets the correct parent branch (e.g., main or release-name) for merge.
[x] Changes are limited to a single goal - eschew scope creep!
[x] Changes are sized such that they do not touch excessive number of files.
[x] All future TODOs are captured in issues, which are referenced in code comments.
[x] These code changes follow the ScubaGear content style guide.
[x] Related issues these changes resolve are linked preferably via closing keywords.
[x] All relevant type-of-change labels added.
[x] All relevant project fields are set.
[x] All relevant repo and/or project documentation updated to reflect these changes.
[x] Unit tests added/updated to cover PowerShell and Rego changes.
[x] Functional tests added/updated to cover PowerShell and Rego changes.
[x] All relevant functional tests passed.
[x] All automated checks (e.g., linting, static analysis, unit/smoke tests) passed.

✅ Pre-merge checklist

[ ] PR passed smoke test check.

[ ] Feature branch has been rebased against changes from parent branch, as needed
Use Rebase branch button below or use this reference to rebase from the command line.

[ ] Resolved all merge conflicts on branch

[ ] Notified merge coordinator that PR is ready for merge via comment mention

[ ] Demonstrate changes to the team for questions and comments.
(Note: Only required for issues of size Medium or larger)

✅ Post-merge checklist

[ ] Feature branch deleted after merge to clean up repository.
[ ] Verified that all checks pass on parent branch (e.g., main or release-name) after merge.

Bug
Get-ApplicationsWithRiskyPermissions does not handle API permissions that belong to apps not listed in the "resources" section of RiskyPermissions.json.

I tested in Test Tenant 3 (E5) and it failed on this line:

I tracked down what was happening and the application being audited had an API permission assigned which belongs to an app that you don't have in the RiskyPermissions.json.

Potential fix
Check if the app id that the permission belongs to is in the "resources" section of RiskyPermissions.json and if not, you can query Graph to find its display name.

Variable name change recommendation
I recommend to rename the variable "Resource" in Get-ApplicationsWithRiskyPermissions. It is not clear which resource we are referring to. Maybe "ResourceAppDisplayName" or something like that because the variable indicates the name of the application that a specific permission belongs to. If you use the same name elsewhere to mean the same thing, then I would change that too.

Bug
In ExportAADProvider where the Get-FirstPartyRiskyApplications and Get-ThirdPartyRiskyServicePrincipals functions are called, I recommend adding parameter validation to ensure that the input parameters RiskyApps and RiskySPs are not $null.
If the preceding functions Get-ApplicationsWithRiskyPermissions or Get-ServicePrincipalsWithRiskyPermissions encounter an error then those parameters will probably be $null and then we probably don't want to call Get-FirstPartyRiskyApplications or Get-ThirdPartyRiskyServicePrincipals.
You should also make sure that the code handles situations where the RiskyApps and RiskySPs are "empty" because your Get-FirstPartyRiskyApplications and Get-ThirdPartyRiskyServicePrincipals functions do not accept empty input objects.
The reason I noticed this is because during testing, Get-ApplicationsWithRiskyPermissions failed due to error but the code proceeded to call the subsequent functions noted above which then generated more errors and error messages.

Your code does not accept empty input objects.

Terms
From what I have seen in multiple web sites, "first party" can refer to applications developed by Microsoft so we might need to use another term to describe apps that the organization running ScubaGear created and owns.

https://learn.microsoft.com/en-us/troubleshoot/entra/entra-id/governance/verify-first-party-apps-sign-in

Permissions to add
Mitchel, I recommend adding the permissions below to RiskyPermissions.json. Please review them and if you agree, go ahead and add them. For any additional permissions we can create a future issue to analyze all the Graph permissions and see if we want to augment or make any further changes.

Application.Read.All (can get information on which apps or service principals can be used to gain access to the tenant)
Organization.Read.All (can read licensing information which might help an adversary determine which security services are enabled during recon)
RoleManagement.Read.Directory (can read which users have highly privileged roles)

Function naming
I am wondering if we should rename the functions Get-FirstPartyRiskyApplications and Get-ThirdPartyRiskyServicePrincipals to more closely align with what they actually do and distinguish them from Get-ApplicationsWithRiskyPermissions and Get-ServicePrincipalsWithRiskyPermissions? Also the term "Get" can imply that a function is retrieving data from somewhere (database, graph, M365, etc.)
Here is what each seems to do:

Get-FirstPartyRiskyApplications - This seems to be an aggregation function.
Get-ThirdPartyRiskyServicePrincipals - This pulls objects from the input parameter RiskySPs that are developed by third parties. It's almost like a filter operation of sorts.

Test Results - Test Tenant 1 (G5)

The code seems to create the JSON in ScubaResults for both applications (owned by the organization running ScubaGear) and third party developed service principals. I went through all the apps and service principals and it seems that the code pulls only the ones that have permissions flagged as high risk in the RiskyPermissions.json file, therefore it is behaving as expected.
The code correctly reports on apps versus third party service principals in the first_party_risky_applications and third_party_risky_service_principals nodes inside ScubaResults.

The code flagged the following numbers of objects as being associated with risk in the tenant:
Applications (owned by the org): 10 objects
Third party service principals: 2 objects
Missing test data
We need some test data for the KeyCredentials, PasswordCredentials and FederatedCredentials fields for objects in the third_party_risky_service_principals node of ScubaResults because right now, the service principals that were flagged have empty data in those fields.

Test Results - Test Tenant 6 (GCC high)
The code fails with what appears to be the same problem as reported in an earlier comment

c

#1471 has been added for future improvements to RiskyPermissions.json

Performance Profiling
I noticed that the addition of the new code appears to add significant time to the execution of the AAD provider so I did some performance profiling. I tried out a couple of potential solutions and will follow-up by email with sample code and specifics. The cmdlet Get-MgBetaServicePrincipalAppRoleAssignment is invoked 720 times when running against the G5 test tenant and that is the root cause of the execution time because each one of those is a REST API call to Graph.
*execution times in screenshots below are in seconds.
Current code performance

Potential solution 1

Potential solution 2


Fix/dbt parse config

Adding fixes to dbt-parse as suggested here in #195 by @jaklan. Note that I've retained add_dbt_cmd_model_args so that I can run dbt-parse like this:
- id: dbt-parse
  args: ["++threads", "64", "--"]

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 97.02%. Comparing base (032cf93) to head (33e3d14).
Report is 8 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #213      +/-   ##
==========================================
+ Coverage   96.95%   97.02%   +0.07%
==========================================
  Files          56       56
  Lines        2624     2622       -2
  Branches      350      359       +9
==========================================
  Hits         2544     2544
+ Misses         59       58       -1
+ Partials       21       20       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@pgoslatara I'm not the specialised in the dbt-checkpoint codebase, but looking at:
def add_dbt_cmd_model_args(parser: argparse.ArgumentParser) -> None:
    parser.add_argument(
        "--model-prefix",
        type=str,
        default="",
        help="Prefix dbt selector, for selecting parents.",
    )
    parser.add_argument(
        "--model-postfix",
        type=str,
        default="",
        help="Postfix dbt selector, for selecting children.",
    )
    parser.add_argument(
        "--models",
        nargs="*",
        help="""pre-commit-dbt is by default running changed files.
        If you need to override that, e.g. in case of Slim CI (state:modified),
        you can use this option.""",
    )

and
def add_dbt_cmd_args(parser: argparse.ArgumentParser) -> None:
    parser.add_argument(
        "--global-flags",
        nargs="*",
        help="""Global dbt flags applicable to all subcommands.
        Instead of dash `-` please use `+`.""",
    )
    parser.add_argument(
        "--cmd-flags",
        nargs="*",
        help="Command-specific dbt flags. Instead of dash `-` please use `+`.",
    )

I believe add_dbt_cmd_model_args should be removed, and your example would look like:

id: dbt-parse
args: ["--cmd-flags", "++threads", "64", "--"]

Both approaches are valid, passing --cmd-flags is more verbose, that's all.
So far, dbt-xxx hooks are being used with --cmd-flags as @jaklan stated, and almost all are loading add_dbt_cmd_model_args  as well.

@BAntonellini the ones which don't use these flags, like dbt clean or dbt deps, doesn't load

@jaklan what do you mean by those hooks don't load?
✗ pre-commit
[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to /Users/bruno/.cache/pre-commit/patch1713464420-77963.
dbt clean................................................................Failed
- hook id: dbt-clean
- exit code: 1

Executing cmd: `dbt clean --project-dir transform`
18:20:22  Running with dbt=1.7.7
18:20:22  Encountered an error:
Runtime Error
  dbt will not clean the following directories outside the project: [.....]

dbt deps.................................................................Passed

These hooks don't load add_dbt_cmd_model_args as these commands don't use any models args - and it should be the same with dbt parse

Now I see, and agree with your point. Although it doesn't "break" anything, having add_dbt_cmd_model_args is useless in this case.
@pgoslatara I'd remove model args, leave only add_dbt_cmd_args and pass any desired flag via --cmd-flags ++some-flag

Now I see, and agree with your point. Although it doesn't "break" anything, having add_dbt_cmd_model_args is useless in this case.
@pgoslatara I'd remove model args, leave only add_dbt_cmd_args and pass any desired flag via --cmd-flags ++some-flag

Thanks for the input on this @jaklan and @BAntonellini, I'm still getting used to this code base so always happy to have a helping hand.
I removed add_dbt_cmd_model_args in 60b1489. What confuses me is that I now need to call dbt-parse like this:
    - id: dbt-parse
      args: ["--cmd-flags", "++threads", "64"]

When I can call dbt-compile like this:
    - id: dbt-compile
      args: ["++threads", "64", "--"]

@pgoslatara isn't that a side-effect of that code?
def main(argv: Optional[Sequence[str]] = None) -> int:
    parser = argparse.ArgumentParser()
    add_filenames_args(parser)
    add_dbt_cmd_args(parser)
    add_dbt_cmd_model_args(parser)
    add_config_args(parser)

and
    def add_filenames_args(parser: argparse.ArgumentParser) -> None:
    parser.add_argument(
        "filenames",
        nargs="*",
        help="Filenames pre-commit believes are changed.",
    )

It seems the parser treats ++threads as a filename and appends it to the final command. You can try to add "--model-postfix", "+" to args, if my assumption is true - that should fail with something like unrecognised argument: threads+ (sorry, I can't test it by myself at the moment).

Hm, maybe I was too fast with the outcome, because filenames don't go through get_flags() which is responsible for replacing + with - - in that case I would expect output like:
The selection criterion '++threads' does not match any nodes
No nodes selected!


Add README.ko-KR.md(translation) in 'data-structures/linked-list'

add README.ko-KR.md(translation) in 'data-structures/linked-list'

Korean translation for the linked list has been added as a part of https://github.com/trekhleb/javascript-algorithms/pull/466. Closing this PR for now, but feel free to update the existing translation if needed.


Improvement/kmyth enclave integration

This merge will integrate the kmyth into the pelz enclave so we can use any functionality it provides directly. It's setup under the assumption that the kmyth repo is a submodule of the pelz repo.

The README needs to be updated to include:
git submodule init
git submodule update

as requirements in order to build the code.

The README needs to be updated to include:
git submodule init
git submodule update

as requirements in order to build the code.

This is addressed in 8560c0f, I also noticed that INSTALL.md may need some other updates to reflect the creation of the pelz-service binary in addition the pelz binary, but I'm not sure what needs to be said about that.


Documentation lacks information to add angular2-jwt for prod

Add the latest version of angular-jwt to the project according to the documentation here:
https://github.com/mgechev/angular2-seed/wiki/Add-external-dependency

Try to call
npm runbuild.prod

Notice, the following error output where the build.bundles.app tries to resolve the angular2-jwt library at /node_modules/angular2-jwt/angular2-jwt.js

To fix the issue, you i.e. have to add something like this within the constructor section of the tools/config/project.config.js file:
    this.SYSTEM_BUILDER_CONFIG.packages['angular2-jwt'] = {
      main: 'angular2-jwt.js',
      defaultExtension : 'js'
    };

Maybe you want to add this information to the affordmentioned documentation (or a better solution than mine. ;) )

Looks good to me! Would you add details to the wiki?

Sure thing. Updated the documentation and added a cast to  to prevent TS lint warnings. Cheers.


Not all pages are being integrated into the PDF

[Enter feedback here] I downloaded the PDF of whole AzureML section but not all of the pages are being integrated. Maybe these are new pages that aren't be scanned or something. FYI.
Examples:
https://docs.microsoft.com/azure/machine-learning/algorithm-cheat-sheet
https://docs.microsoft.com/azure/machine-learning/how-to-select-algorithms

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5cfad9fe-242d-c81a-319a-393ace5480a5
Version Independent ID: fad36df8-e42c-3246-3fde-388f4bc670ba
Content: Machine Learning Algorithm Cheat Sheet - designer - Azure Machine Learning
Content Source: articles/machine-learning/algorithm-cheat-sheet.md
Service: machine-learning
Sub-service: core
GitHub Login: @FrancescaLazzeri
Microsoft Alias: lazzeri

Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

Hi @residentx10 ,
Thanks for the report. I'm investigating to see why the PDF document isn't including some of the articles in the document set.

#assign:@peterclu

@residentx10, I've submitted a PR that should resolve the issue of the PDFs not rendering properly. You should see the changes live in a few days. I'll proceed to #please-close this issue.
If the issue persists even after a few days, please let me know by opening another issue and adding me with @PeterCLu .

@PeterCLu Thanks. I noticed the document sets for AML is getting bigger too.
On Friday, I redownloaded the pdf set for AML but I still had problems.
On Android in the Adobe Reader, I cannot move to pages from the "content" window.
I'm on the road a lot so I download the docs set and read it on Android Go Tablet to make reading easier.
Should I uninstall the android reader and just read in a browser? I'll attached some screenshots to show what I mean.

Image Screenshots:
Location in Adobe Reader

Marked Issues in Adobe Reader

Android GO Version 9
Adobe Reader Pro <IP_ADDRESS>77


Fix typo

/github action/i -> GitHub Actions

@changeworld : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"

@changeworld - thank you for your contribution.  We use "GitHub Actions" to refer to the service/feature, we use "GitHub action" to refer to an instance of GitHub Actions.
#please-close.

@mumian
I understand.


Repeating error

3x GogoLoot\loot-Loot.lua:907: attempt to index global 'UnitPopupButtons' (a nil value)
[string "@GogoLoot\loot-Loot.lua"]:907: in function <GogoLoot\loot.lua:783>

Locals:
capitalize = <function> defined @GogoLoot\loot.lua:784
printInvalid = <function> defined @GogoLoot\loot.lua:787
events = <unnamed> {
 0 = <userdata>
}
canLoot = true
lootAPIOpen = false
lootTicker = nil
lastItemHidden = false
(*temporary) = nil
(*temporary) = <table> {
 color = <table> {
 }
 text = "Epic"
 checkable = 1
}
(*temporary) = <table> {
 hex = "|cffa334ee"
 r = 0.639216
 color = <table> {
 }
 g = 0.207843
 b = 0.933333
}
(*temporary) = "ITEM_QUALITY1_DESC"
(*temporary) = "ITEM_QUALITY2_DESC_GL"
(*temporary) = "ITEM_QUALITY3_DESC_GL"
(*temporary) = "ITEM_QUALITY4_DESC_GL"
(*temporary) = "CANCEL"
(*temporary) = "attempt to index global 'UnitPopupButtons' (a nil value)"
debug = <function> defined @GogoLoot\loot.lua:1

This error seems to be repeating, I've not looked at it myself but if I find a fix after the raid I'll make a PR!

Maybe switch to an add-on that's still being supported. Cheers!

@sfjuocekr I've worked to ensure Gargul handles most all the same features that GogoLoot offered.
I just have no desire to support this, the world can't have multiple add-ons that interact with the loot frame that run at the same time... and this project stopped being fun for me.
https://www.curseforge.com/wow/addons/gargul

Done!


Fix old redis-developer links in README

When browsing through the README, I noticed some of the links were based on an old code location, so I have updated those in this PR, and the PRs linked below for each repo:
redis/redis-om-dotnet#46
redis/redis-om-node#37
redis/redis-om-python/pull/95
redis/redis-om-spring#14

@dracco1993 - please just add your handle to the Contributors section of the README, otherwise LGTM.

@slorello89 I've added it in d1df3b8  🙂


Connections during import time makes testing hard

During testing, it's ideal that a Neo4j server is not required for the tests to pass. Neomodel makes this really hard by requiring the server during import time.
Is there any way to work around or disable this feature?

I've worked around the issue by monkey patching `core.install_labels' in my test module before importing any neomodel references.
# Monkey patch install_labels to disable connecting to Neo4j during import time
# in tests
def patch_install_labels(cls):
    pass

core.install_labels = patch_install_labels

I've added a setting for this https://github.com/robinedwards/neomodel/tree/master/doc/source


Example app: "release" channel button

This PR fixes the "Release" channel button. Previously, if a user released a channel, the state of the channel would no longer be updated on the UI. Now, you can detach or release from the channel, and still attach later.

I'm a little concerned that this PR conflates a library implementation fix (new call to super in lib/src/platform/src/realtime/realtime_channels.dart) with example app fix, but given @ikbalkaya and @lukasz-szyszkowski have approved it then I'll assume this has been explained, or at least understood by someone with more context than I.


(PUP-3839) Use PL Gem mirror in acceptance tests

The testing infrastucture should be resilient to network issues. For
acceptance tests that exercise the Gem provider and setup that installs
Gems, there hasn't been a clean way to specify that the internal mirror
should be used. This follows the Gemfile pattern and selects the
internal mirror if it's available.

@MikaelSmith it looks like that beaker PR has been merged. Should this be updated to use that new facility? (I only looked at it briefly so don't know much.)

@rji acceptance/Gemfile has extra logic for defaulting to our rubygems mirror unless a envvar is specified; I copied that logic for use in the acceptance setup. So I could change to something like:
source ENV['GEM_SOURCE'] || (ENV['OUTSIDE_PUPPETLABS'] ? 'https://rubygems.org' : 'http://rubygems.delivery.puppetlabs.net')

This is a matter of personal preference, but I don't particularly like defaulting to internal resources. I'd prefer the default be public rubygems.org, and if $GEM_SOURCE is set then use the mirror URL provided. I still don't know where OUTSIDE_PUPPETLABS came from; we don't actually set that anywhere.
In the end, I'm okay with either approach, just as long as we're consistent. @kylog or @jpartlow can make the final call here.

@MikaelSmith I'm having a little trouble figuring out what the changes are in comparison to the whitespace shift in install_utils; are the significant changes in configure_gem_mirror?

Yes, the main change is adding and using configure_gem_mirror. You can remove the whitespace shift using the 'No Whitespace' checkbox.

There's a no whitespace checkbox? /me stares at his screen.

There's a no whitespace checkbox? /me stares at his screen.

@jpartlow I'm guessing @MikaelSmith is using https://github.com/KuiKui/Octosplit. If not, I recommend it :)

@MikaelSmith I thought a little bit more about this over the weekend, and I think @rji is right that we should default to the rubygems.org that will work everywhere, and not force folks running acceptance outside of Puppetlabs to figure out that they have to specify the OUTSIDE_PUPPELABS env variable.
For internal use, we should just rely on setting GEM_SOURCE if we want the internal mirror.
So I'm in favor of dropping the extra logic checking OUTSIDE_PUPPETLABS, and perhaps updating https://github.com/puppetlabs/puppet/blob/master/docs/acceptance_tests.md to note the use of GEM_SOURCE and our internal rubygems mirror if you are manually testing inside the VPN.

Ok, I'll make that happen.

It also seems like sqa-utils belongs in a Gemfile.local or Gemfile.global, not the project Gemfile. I'd have to specifically check for using our internal server to leave it in, so I'm opting to remove it.

Sigh, sqa-utils, right; so that's needed if you are manually running the bin/ci-pe-puppet.sh (manually running acceptance for the pe-puppet repo).  It could certainly be put in a Gemfile.local, but you would need to know to do that.  The bin/ci-pe-puppet.sh script has some basic help output, and this information could be appended there.  (The current jenkins jobs for pe-puppet rewrite the Gemfile before running...)

So we'd need to get the jenkins jobs updated before merging...

No, the pe-puppet Jenkins jobs create their own Gemfile and add sqa-utils before running, as I recall.  The sqa-utils reference is only needed if you are manually running acceptance against a pe instance using the bin/ci-pe-puppet.sh script.  You just need to know that you need it, which is why I suggested updating the help in that script to let people know they need to add it to their Gemfile local.

Ok, will do.

@MikaelSmith sounds like all this needs is the tweak to ci-pe-puppet.sh's help wrt adding sqa-utils to Gemfile.local. (And TIL there is a sqa-utils gem ....)
Other than that :+1:

The test already does bundle install, I could also have it add sqa-utils to Gemfile.local itself.

Yeah, or just document that the user should do the same. No strong feelings.

:+1:


Refresh CORD-19 annotations with latest scibite annotations

https://github.com/SciBiteLabs/CORD19/issues/14
Note they use PRO as the vocabulary. We can ingest the PRO file... but it may be easier just to add the PRO xrefs into our curated gpi file

@cmungall @deepakunni3 does SciBite definitely use PRO as their vocabulary for SARS-CoV-2 genes? I can't seem to find for example the PRO ID for Spike protein:
https://proconsortium.org/app/entry/PR%3A000050269/
in our transformed data:
$ grep 000050269 data/transformed/SciBite-CORD-19/nodes.tsv  data/transformed/SciBite-CORD-19/edges.tsv
$

Closed by #263


Can this support type check with typescript and rewrite all Class into es6 mode?

Type check with typescript
When I use npm package, I found it is no code intelligence like other libs.
We can only use typescript to generate the d.ts declaration file from the existing jsDoc comments, instead of rewriting all the js files into ts files. In this way, you can get a better development experience, that is, the code intelligence hints.
Rewrite class mode from constructor function to es6 class
I read source code and also found that some source code such as LonLat.js, Extents.js, etc. was written in es5 constructor function mode, not es6 class.
So I would like to rewrite them into es6 mode, it looks more modernly.
The jsDoc comments will remain intact.

FYI @Zemledelec @pavletto

I think d.ts is a good idea.

The first reason I left .prototype. is that I'm not sure that classes works faster in js. Modules (or classes) such as LonLat, Extent, Segment etc. are critical for engine. So, before making a decision we must be sure that this or those approach doesn't make any negative affect on the performance of the engine.
The first step to make the decision is to prepare proper tests which shows that the classes signatures are satisfied the condition above.

Replaced with https://github.com/openglobus/openglobus/issues/375


[Bug]: Setting APPSMITH_SERVER_TIMEOUT bigger than 60sec do not help with query running longer thant 60sec

Is there an existing issue for this?

[X] I have searched the existing issues

Description
I need to increase the timeout with Oracle datasource.
We set APPSMITH_SERVER_TIMEOUT=1800, but it fails with this error after 60sec:
Deployement on Kubernetes using bitnami chart.
backend container error:
boundedElastic-2: Execute Error closing Oracle StatementClosed Connection
frontend container error:
023/11/16 09:54:29 [error] 111#111: *209 upstream timed out (110: Connection timed out) while reading response header from upstream,
client: xxxx server: _, request: "POST /api/v1/actions/execute HTTP/1.1", upstream: "http://xxx/api/v1/actions/execute", host: "xx", r
eferrer: "http://xxxx/app/my-first-application/page1-6555e3eb2e0478728a9497f7/edit/queries/6555e43a2e0478728a9497fd?editName=true&showTemplate=false&from=datasources"
Steps To Reproduce
in the oracle db, create the following function:
create or replace function func
return number
as
begin
dbms_lock.sleep(80);
return 0;
end;
/
create a dummy oracle query:
select func from dual
run the query from appsmith
Public Sample App
No response
Environment
Release
Issue video log
No response
Version
Selfhosted, kubernetes, v1.9.44

@ChriRene99 did you change the timeout settings in the query?

yes of course, the timeout in the setting query has been set to something bigger than 60sec.
In the gui, I dont get timeout error but: An unexpected error occurred

What is the status of this ticket?

This issue is occurring due to READ TIMEOUT configuration not present in Caddy config.
https://github.com/appsmithorg/appsmith/issues/37244
Assigning this issue to the Devops team.


ARROW-12489: [Developer] autotune is broken [WIP]

@github-actions autotune

@github-actions autotune

Did you confirm this on your fork?
You can confirm this on your fork by the following:

Push this change to your fork's master
Create a test pull request on your fork
Comment @github-actions autotune on the test pull request

Yup, it works on my fork: https://github.com/jonkeane/arrow/runs/2422954470?check_suite_focus=true


Unable to install using goinstall

Trying to install gocode using goinstall, one gets the following error message:
$ goinstall github.com/nsf/gocode
goinstall: : path "~/configfile" not inside a GOPATH

Works with go tool now. See updated README.md for instructions, shortly: go get -u github.com/nsf/gocode


how can I put the y-axis on the right?

Hello,
I could not find out how to make the y-axis of a Curve chart appear on the right instead of on the left.
Is that possible?
Thanks,
Michel

Using hv.help you can find out what options apply to an Element. To position the axis to the right you can do something like:
%%opts Curve [yaxis='right']
hv.Curve([1, 2, 3])

or using the .opts syntax:
hv.Curve([1, 2, 3]).opts(plot=dict(yaxis='right'))


Added force inlining to device_merge_sort_mergepath

Added the missing ROCPRIM_FORCE_INLINE directive to the block_merge_kernel

Missing this caused out-of-local-memory errors in debug configuration

Formatting changes

There appear to be test failures with device_reduce_by_key with MI200 + hipMallocManaged, but I don't see how that is related to any of these changes.  Let's merge this in for now to get pytorch debug builds working again, I'll take a closer look at the MI200 + HMM failures later.


[BUILD] Update scalatest.exclude.tags

Why are the changes needed?

Hudi does not support Spark 3.2 yet.
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

thanks, merging to master


installation fails  in recent RedHat/Fedora environments

On Fedora 21 pip install musicplayer fails with a compile error due to wrong include path:
PlayerInStream.hpp:9:34: fatal error: libavformat/avformat.h: No such file or directory

The desired files are installed via the ffmpeg-devel package but they are located under /usr/include/ffmpeg/libavformat/
As reported for other projects this seems to be not a bug in the package but a change in the ffmpeg-project, see here:

https://github.com/dirkvdb/ffmpegthumbnailer/issues/18
https://ask.fedoraproject.org/en/question/31642/how-to-install-libav-devel-packages-properly

I didn't try it but simply adding /usr/include/ffmpeg to the include paths might (quick)fix this.

I suspect I have a somewhat similar problem using Ubuntu 14.04 LTS ...
after trying the following I gave up in confusion.
sudo easy_install musicplayer
Searching for musicplayer
Reading https://pypi.python.org/simple/musicplayer/
Best match: musicplayer 1.20141030.091600
Downloading https://pypi.python.org/packages/source/m/musicplayer/musicplayer-1.20141030.091600.tar.gz#md5=489e2cf4dc28bc35bda13f9c9b38e984
Processing musicplayer-1.20141030.091600.tar.gz
Writing /tmp/easy_install-Hct1wr/musicplayer-1.20141030.091600/setup.cfg
Running musicplayer-1.20141030.091600/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Hct1wr/musicplayer-1.20141030.091600/egg-dist-tmp-kwGItz
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from musicplayer.h:38:0,
from musicplayer_getmetadata.cpp:9:
PyThreading.hpp:7:33: fatal error: boost/noncopyable.hpp: No such file or directory
#include <boost/noncopyable.hpp>
^
compilation terminated.
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
sudo apt-get install python-dev libsnappy-dev libtool yasm libchromaprint-dev portaudio19-dev libboost-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-dev is already the newest version.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies.
portaudio19-dev : Depends: libjack-dev
E: Unable to correct problems, you have held broken packages.
sudo apt-get install libjack-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies.
libjack-dev : Depends: libjack0 (= 1:0.121.3+20120418git75e3e20b-2.1ubuntu1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

I made a fix for fedora 23 : https://github.com/albertz/music-player-core/pull/10
dnf install ffmpeg-devel portaudio-devel libchromaprint-devel

It sounds like this is fixed. Closing now. Please reopen if the problem is still there, or open another issue if there is another problem.


Trigger a function upon connection

Upon connection, I would like to call a function, only if I specify that function.
See redis-py: on_connect

After discussion - not relevant in this client.


Upmerge 01.03.2024

Zephyr upmerge 01.03.2024

With the latest push the history of the upmerge branch was rewritten to align with the history cleanup on main.

CI failure checks, manual:
Coding guidelines: OK
Scancode: OK
West Command Tests: Unknown, likely due to upstream CI config


Hardcoded paths (noise + excised tim)

There are a few places where the directory structure and naming convention for the noise chains are hardcoded. There are a few separate expectations for what the path is and the files are called around the place, these are the ones I found.
in noise_utils.add_noise_to_model
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L277-L284
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L291
in noise_utils.analyze_noise
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L53
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L56
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L62
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L65
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L108-L111
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/noise_utils.py#L133
in utils.check_recentness_noise
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/utils.py#L1235-L1240
Also the path to the excised TOAs is hardcoded in
utils.check_recentness_excision
https://github.com/nanograv/pint_pal/blob/50592396777518c49439a3e154c2d852efe883b7/src/pint_pal/utils.py#L1270-L1275

Some of these, such as the use of chain_1.txt and pars.txt, are likely to be unproblematic, since the filenames are relatively standardized at the enterprise level. Others may cause problems, but it's likely we'll be changing the way we approach the noise analysis for NG20 anyway. We should revisit this once we have a better idea of how that will work.


Fixing "Archive is corrupt" while unpacking added to ZIP null-bytes files.

To fix this issue, just add in "Store":
if (_source.Length == 0)
{
_zfe.FileSize = 0;
_zfe.CompressedSize = 0;
_zfe.Crc32 = 0;
_zfe.Method = Compression.Store;
return _zfe.Method;
}

Implemented as suggested


[Bug] Search Issues

I can't filter on anything past the first few parts of the title when searching.
for example:
PRODUCTION DEGRADED: ptc0231.xl000_ptcprd085.wla.tla.uprr.com (CHK_LOG) - LOGS is degraded - ptcprd085.wla.tla.uprr.com - Workload Job PTC0231.XL000 is in FAILURE status
searching for production or degraded returns this item, trying to filter on anything else (like logs, or workload) returns nothing.

This has been patched under PR #129 and merged into develop - closing issue.


Bug: fix Docusaurus blog RSS feed limitations

🐛 Bug Report
With https://github.com/facebook/docusaurus/pull/4330, we now render the full blog posts in the RSS feed, as HTML.
The way we currently do it is "good enough" but has several limitations:
MDX components replaced by divs
Using this MDX:
# title

import SomeCompent from "@site/src/someComponent"

<SomeCompent>Hello</SomeCompent>

Will render as:

title

Hello

The logic of <SomeComponent> is never evaluated.
For layout/wrapper components, it's not a big deal as the content remains in the RSS feed, but in some cases, it can be worst.
Not having a global MDX config and giving themes the ability to "register" some MDX comps and scope can help solve this problem. Related to https://github.com/facebook/docusaurus/issues/4625
No  ```mdx-code-block support
 ```mdx-code-block is an escape hatch we added so that external tools (like Crowdin) do not mess-up with unsupported MDX markup.
The MDX content in these code blocks must be evaluated, as if it was not a code block
No relative image path support
MD referencing co-located assets such as:
![Docusaurus beta party](./img/image_cropped.png)

will be transformed as

Unfortunately, the image is not hosted at ./img/image_cropped.png in the production site due to using Webpack's file-loader.
The real image URL in prod is /assets/images/image_cropped-2744c01b1fc0b451dbdb65a1d568fbff.png
We need a way to ensure such MD image declarations work in the RSS feed properly (ideally without changing the existing image asset URL or duplicating the asset in final site build)

Regressions that make sites fail to build at all:

styles is not defined when import styles from './styles.module.css';
require is not defined when require('./assets/img.png').default.

The missing Webpack infra is more severe than missing MDX features

You do need to have the metadata about blog posts again, and where they were rendered. And a class or ID on the body of the text. But I don't really find it hack if it's then very close to what instapaper or safari reader or so do, too.
Alnternatively: do it when you render the content into the body. Perhaps render it twice, once with a "this is for the static website" and once with "this is for the static feed", so components can switch and return different markup for each "media query"?

Not so critical bug: #6195