test(yjs): Spike

closes #68
This is not to be considered production ready code or anything to merge. This is supposed to be an example on how yjs works.
Learnings:

Integrating y-websocket into the existing next.js server seems to be cumbersome, at least I struggled with a simple problem: the rooms are expected to have a url like /socket/ROOM_A. Unfortunately next.js uses its own logic and is searching for a module with the same name (ROOM_A) for routing...didn't figure out how to deactivate that. Maybe rewrites help.

At least for http requests its: /api/room/[id]/route.ts - currently not working for web sockets via next-ws.

y-websocket updates do not forward the whole state on each update. So it's not as easy as just storing updates each time in the database. We would need to write an algorithm (can be as simple as saving every x minutes the whole state) when to only forward diffs to other users and when to save the whole document.
So I tried y-redis which is now integrated in this PR. It does two things: 1) serves as y-websocket backend, 2) caches the smaller diff updates in redis and then later stores the whole document in Postgres. This is configurable. So takes care of the whole process. The y-redis server is, however, too complicated for us as its also implements an auth layer before entering a room via JWTs. I don't think we need that, so we will need to throw all of this out. For this spike, I hacked my way around it.
I have used quill editor with the existing bindings for yjs. All easy to integrate. Sure you can write your own editor and binding, but this one is does it all for no work at all...guess its also possible to extend it.

Instructions
You need redis, Postgres and the branch => Better use my docker compose setup.
docker compose up -d
docker compose exec app npm run dev

EDIT: and please update the .env file, some new additions there.

@SamTV12345 @AugustinMauroy feel free to test

It's possible to bind the editor with our custom tools bar ?

It's possible to bind the editor with our custom tools bar ?

you have to write your own binding. yjs comes with bindings for the popular editors, if you want to build a custom one, you need your own bindings.

Actually some people just answered my question:/api/rooms/[id].js would be the way to go, haven't tried it yet with the next-ws stuff.

I didn't 100% catch you issue but. if you do something like that /api/rooms/[id]/route.ts next will compile

Actually some people just answered my question:/api/rooms/[id].js would be the way to go, haven't tried it yet with the next-ws stuff.

I didn't 100% catch you issue but. if you do something like that /api/rooms/[id]/route.ts next will compile

yes, would also be possible. would leave the logic when to store the whole document and when the diff to us. We could choose if we want to go the redis or the custom backend path...both possible I guess. redis is more complicated as setup but the algorithm is already there.

yes, would also be possible. would leave the logic when to store the whole document and when the diff to us. We could choose if we want to go the redis or the custom backend path...both possible I guess. redis is more complicated as setup but the algorithm is already there.

It's good that you said it again. But if the instance doesn't use docker (for example a simple VPS) this will add complexity to the setup. How is this currently done? And again, what is the maintenance cost? Because a custom algo isn't death.
Also, from a performance point of view, you have to look at whether you're running a full redis or just running a bit of JS when you need it (normally nextjs handles this well).

yes, would also be possible. would leave the logic when to store the whole document and when the diff to us. We could choose if we want to go the redis or the custom backend path...both possible I guess. redis is more complicated as setup but the algorithm is already there.

It's good that you said it again. But if the instance doesn't use docker (for example a simple VPS) this will add complexity to the setup. How is this currently done? And again, what is the maintenance cost? Because a custom algo isn't death. Also, from a performance point of view, you have to look at whether you're running a full redis or just running a bit of JS when you need it (normally nextjs handles this well).

there are no perfect solutions, just tradeoffs. Yes, on a simple vps it may be difficult, but most are also offering databases and redis. And even without, it's still possible. We will need to implement the web socket backend part ourselves which is currently handled by y-redis. As a reminder: the scope of this PR was not to offer a perfect solution. the scope of this pr was to get a feeling for yjs and see it in action. I am happy to try out another way of implementing yjs, but there is no silver bullet here.

Because a custom algo isn't death.
I strongly disagree here, even if it's not death. Merging conflicting updates in a reliable way is complex, time consuming to code, prone to errors and results in higher code complexity. And this is completely solved within yjs even with all the editor bindings. Why reinvent the wheel here if you get it for free?

@AugustinMauroy I pushed a commit where I am trying to connect to an API endpoint in next.js api/room/room_id which is located here: api/room/[room].ts. However, I still get the message:

[next-ws] could not find module for page /api/room/quill-demo-room-websocket-server

Please feel free to play with it. Not sure why it can't find the module, either I am doing something wrong or next-ws has problems supporting dynamic paths? Unfortunately I don't have more time today.

ok last update: the GET request is reaching the api endpoint, but not the web socket request...

ok last update: the GET request is reaching the api endpoint, but not the web socket request...

I'll take a look it's shouldn't do that. EDIT:

try to use next-socket as client such as app/socket-demo.
There are a strange behavior, when you are on dev env sometime it's not compile socket route but on build it's work fine.

well, this would make using the existing bindings more complicated (as you can't simply use the yjs web socket adapter) and I honestly do not really understand why the path resolutions isn't working as the correct route is called. But would be interesting to see if that actually works. But I wouldn't consider this for real use.

ok last update: the GET request is reaching the api endpoint, but not the web socket request...

I'll take a look it's shouldn't do that. EDIT:

try to use next-socket as client such as app/socket-demo.
There are a strange behavior, when you are on dev env sometime it's not compile socket route but on build it's work fine.

well, this would make using the existing bindings more complicated (as you can't simply use the yjs web socket adapter) and I honestly do not really understand why the path resolutions isn't working as the correct route is called. But would be interesting to see if that actually works. But I wouldn't consider this for real use.

The question is what would be the benefit of using next-socket over the yjs implementation.

ok last update: the GET request is reaching the api endpoint, but not the web socket request...

I'll take a look it's shouldn't do that. EDIT:

try to use next-socket as client such as app/socket-demo.
There are a strange behavior, when you are on dev env sometime it's not compile socket route but on build it's work fine.

well, this would make using the existing bindings more complicated (as you can't simply use the yjs web socket adapter) and I honestly do not really understand why the path resolutions isn't working as the correct route is called. But would be interesting to see if that actually works. But I wouldn't consider this for real use.

The question is what would be the benefit of using next-socket over the yjs implementation.

i don't see any but maybe it has to be used so that the routes work properly, we have to test that. Currently I see the following options:

We make the routes work, so that we can integrate the web socket backend into the next.js server. Currently stuck because the route is not picked up from web socket calls to next-ws, but normal http requests are. I agree that on first glance this seems to be the most straight forward solution, but we need to make next-ws work. => Help wanted
We bring our own small dedicated web socket server (there is already a template in the y-websockets bin folder) that is only serving as a yjs backend. So 2 backend services are running in the end. Also means two services need to share a database connection or we need 2 databases.
Integration via y-redis. This already works properly but requires a redis which might be too complicated for smaller etherpad setups. For bigger installations, this might be actually better than the other solutions.

We make the routes work, so that we can integrate the web socket backend into the next.js server. Currently stuck because the route is not picked up from web socket calls to next-ws, but normal http requests are. I agree that on first glance this seems to be the most straight forward solution, but we need to make next-ws work. => Help wanted

I'm more in favor of that. It's more custom things but it's reduce number of backend. We can ask help from maintainer of next-ws ?


Photo Roster

The photo view should be a PDF just like this PDF:  photoview_sample.pdf
The attached PDF was pulled from the "Print" function on the photo/ view on our environment.

The list view CSV download looks fine.

Priority = HIGH

This is going to take a while because there is no document to start with. The export from the roster tool cannot be used. I'll have to build a HTML file and then PDF it. Is the PDF essential or would be HTML file be ok?

@steveswinsburg - yes an HTML is fine. Since this is one of the highest priority items, can you work on this before the LOW and MEDIUM items, please?

OK I have a design for this and this is up next. I needed to get a few of the architectural things done first, mainly #63 and the associated tasks for that so that both the custom photo tool and roster could coexist.

This is nearing completion and will be on m6.

@cacostantino OK I am going to push up the first cut of some custom HTML for the photo roster export. Since we don't have actual photos (integrates with the Yale Photo Directory), everyone gets the same photo, however it should be rendering in a grid with image and then the data fields underneath. Sample file attached (it's a PDF but the file is actually HTML, I can't attach HTML here)
roster.pdf

Yes, this looks pretty good - assuming the missing data will display once we can get this into Classes*v2 to test.

@cacostantino, for 1, my test data doesn't have those fields so they don't show. Do you want them to show even if there is no data?
2 and 3 I am addressing now.

Done. Will push to the AWS server once I have a few more things done.

OK this should be on the AWS server now.

@steveswinsburg  - it's looking good, but I want to see what it looks like with our actual data.

Roster-export changes needed:

Name - student names should be in alphabetical order
Col - current data is "section" which is not needed; correct data is currently listed under Major.
Major - current data is "Col"; correct data is currently listed under Year.
Year - current data is "Major"; correct data is currently listed to the right under no heading.
Last column with no heading and number data is not needed.

Photo roster HTML changes needed:

Name - student names should be in alphabetical order
Col - current data is "section" which is not needed; correct data is currently listed under Major.
Major - current data is "Col"; correct data is currently listed under Year.
Year - current data is "Major"; correct data is currently listed to the right under no heading.

Discussed this with Asha a few weeks ago. Your roster code is missing a
line of code from the merge, which means the column headers are out of
whack. Asha was going to resolve this.
This data comes directly from the excel export. What order are the names in
in the excel export?
On 19 Sep. 2017 03:43, "Christine A. Costantino"<EMAIL_ADDRESS>wrote:

Photo roster HTML

Name - student names should be in alphabetical order
Col - current data is "section" which is not needed; correct data is
currently listed under Major.
Major - current data is "Col"; correct data is currently listed under
Year.
Year - current data is "Major"; correct data is currently listed to
the right under no heading.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/steveswinsburg/sakai-archiver/issues/85#issuecomment-330299149,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAH4aYiiTMgwIge2cYUAcyTLNc5BihOtks5sjqu2gaJpZM4OJzKv
.


Frozen Columns

Why all columns are frozen columns and not only the data type?

?

By default, elasticsearch document field are multivalued, so each document field of type X is mapped to a cassandra column of type list (where X is a native type string, int, long etc...).
If you want to map field of type X directly to a cassandra column of type X, set the mapping attribute cql_collection to singleton. For example,
curl -XPUT "http://:9200$NODE/twitter/_mapping/tweet3" -d '{
"properties" : {
"size" : {
"type" : "long",
"cql_collection":"singleton"
},
"post_date" : {
"format" : "strict_date_optional_time||epoch_millis",
"type" : "date",
"cql_collection":"singleton"
},
"message" : {
"type" : "string",
"cql_collection":"singleton"
},
"user" : {
"type" : "string",
"cql_collection":"singleton"
          }
        }

}'
create the following cassandra table.
CREATE TABLE twitter.tweet3 (
"_id" text PRIMARY KEY,
message text,
post_date timestamp,
size bigint,
user text
)
If you want to map field of type X to a unique list of values (a set), set cql_collection to "set".
Does this answer your question ?
Le 26 févr. 2016 à 09:47, Chris Feldmeier a écrit :

Why all columns are columns and not only the data type?
—
Reply to this email directly or view it on GitHub.

yeahh cool thanks :+1:


0.0.9 version doesn't compile when cocoapods is used

I added BKMokeyKit with help cocoapods. And when I try compile my project I have error:
BKMoneyKit/BKForwardingTextField.m:15:1: Cannot synthesize weak property because the current deployment target does not support weak references
If I change Deployment Target in BKMoneyKit my project will compile.
Cocoapods version is 1.1.0

What Xcode version do you use? And what's projects minimum OS target version?
나의 iPhone에서 보냄

오후 8:05 YuriyNovikov<EMAIL_ADDRESS>작성:

I added BKMokeyKit with help cocoapods. And when I try compile my project I have error:
BKMoneyKit/BKForwardingTextField.m:15:1: Cannot synthesize weak property because the current deployment target does not support weak references
If I change Deployment Target in BKMoneyKit my project will compile.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@bkook
I use Xcode 8.0.
Minimum OS target version 8.0.

In your screenshot, iOS Deployment Target is 4.3.
Can you try changing it 5.0 or above?

@bkook  I can change but this file was generated by cocoapods. I think you have to add ios.deployment_target in your podspec file.

+1

@YuriyNovikov @mcany
Sorry for late. I fixed as you said just now. Please check version 0.0.10.


[front] Add read checks

Description
Additional canRead() check on endpoints
Risk
Some queries may be blocked
Deploy Plan
deploy front

Seeing the duplication and the risks of omitting checks in other endpoints, it might be worth using the resource wrappers, such as the withDataSourceFromRoute you recently added, to enforce this checks no? would that work?

what I meant here would be to generalize withResourceFromRoute so it can take multiple resources, and require the caller to specify the permissions to check on those, e.g. withResource**s**FetchingFromRoute(handler, { "space": ["read", "write"], "datasource": ["read"} }
wdyt?

cify the permissions to check on those, e.g. withResource**s**FetchingFromRoute(handler, { "space": ["read", "write"], "datasource": ["read"} }
wdyt?

👍
we are discussing it here :  https://dust4ai.slack.com/archives/C05B529FHV1/p1734088226745539?thread_ts=1733948732.333009&cid=C05B529FHV1
but should be done in another PR


feat: Reanimated V3

What
Support for Reanimated V3 and the new Worklet runtime! This allows copying in Host Objects from outside context (with help from @tomekzaw)

Changes

Tested on

Related issues

Hello & thanks for your library.
Following my question about react-native-reanimated, I was redirected to this PR.
I would like to know if react-native-camera will work with react-native-reanimated v3 and react-native 0.72 ? 😊
When I add this PR to my package.json, I get this error :

frame-processor/unavailable: Frame Processors are not enabled. See https://react-native-vision-camera.com/docs/guides/troubleshooting, js engine: hermes

"react-native": "^0.72.1",
"react-native-reanimated": "3.3.0",
"react-native-vision-camera": "https://github.com/mrousavy/react-native-vision-camera.git#feat/rea-v3",

Thank you in advance for your answers :)

Hey - sorry I am currently working on too many things at once, V3 is more important for me right now and I think Reanimated V3 has some breaking changes that made it hard for me to add it to VisionCamera V2 so maybe I need to wait for them to fix some stuff first before I can use it.

@mrousavy thank you for your reply and your work :)

yep, confirming what @mrousavy said, I'm currently working on integrating RNVisionCamera with Reanimated v3.
Spoiler: it already works, just doing some cleanup now 🚀

woohoo tomasz ur the man

yep, confirming what @mrousavy said, I'm currently working on integrating RNVisionCamera with Reanimated v3.
Spoiler: it already works, just doing some cleanup now 🚀

woohoo! waiting on this :)

yep, confirming what @mrousavy said, I'm currently working on integrating RNVisionCamera with Reanimated v3.
Spoiler: it already works, just doing some cleanup now 🚀

woohoo! waiting on this :)

+1 💪

I've found a temporary workaround (kinda). I install react-native-reanimated using an alias. e.g.  pnpm i rnra3@npm:react-native-reanimated@latest
I can then import ... from 'rnra3'

Hello @tomekzaw ,
Do you have an approximate release date for this PR ? :)
Many thanks in advance :)

Really appreciate this PR!
We also require it for a feature using vision-camera-code-scanner.
Happy to help with testing or anything else to get it released

@markrickert Thanks a lot for your comment! I'm glad to hear that you use it!

Android, however got a build error

That's weird, it works for me but I will try to replicate your setup and see if I can reproduce this error.

this has to to with the pre-built binaries in the reanimated nightly

Reanimated 2 used to be distributed as precompiled binaries but it is no longer the case – Reanimated 3 is always built from source.

So we're getting closer, but not quite there yet.

Exactly. There's still some minor tweaks to be done on Reanimated side and some cleanup on RNVisionCamera side but I think I can get this working tomorrow.

I will try to replicate your setup and see if I can reproduce this error.

I think you can reproduce this by spinning up a new react-native project (I'm on 0.17.13 currently, adding the two dependencies i listed in my previous comment, and then ensuring your minSdkVersion in the build.grade file is set to 23 and not 21.
Please let me know if i can be of further assistance and thanks again for working on this!

Hey @markrickert,

I think you can reproduce this by spinning up a new react-native project (I'm on 0.17.13 currently, adding the two dependencies i listed in my previous comment, and then ensuring your minSdkVersion in the build.grade file is set to 23 and not 21.

Thanks for these instructions, I was able to successfully reproduce this error: User has minSdkVersion 21 but library was built for 23 [//react-native-reanimated/reanimated]
The problem actually comes from RNVisionCamera since in its android/build.gradle there's minSdkVersion 21 (see here). However, Reanimated reads the settings from the root project (see here) so effectively it's build for minSdkVersion 23. As a workaround, you can simply patch minSdkVersion in react-native-vision-camera/android/build.gradle. I think it deserves a separate issue and a PR with the fix.

When i changed my project to 21, the app compiled and launched correctly! However, with zero code changes to our barcode scanning screen, the app will crash after about 5 seconds.

I've analyzed the attached stack trace and it looks like facebook::jsi::Function::callWithThis fails for some reason. I've replaced the call with WorkletRuntime::runGuarded in 6b2cb36b60cd4994e66ce85d61efd952e657b57d. Can you please check if the following setup works for you? It works fine for me on Android: https://twitter.com/tomekzaw_/status/1697604643365278077
    "react-native-reanimated": "software-mansion/react-native-reanimated#5b93afe19c5451f67c3667efe1b6497daff54d4a",
    "react-native-vision-camera": "tomekzaw/react-native-vision-camera#db4ff0bd1bc2b1a2a8d11f1fb3dcba3e06132c68",


Can't build application with cpprealm library

Hello,
Recently found cpp implementation for Realm and tried to build test program, but failed.
I have latest OpenSUSE Tumbleweed.
-- Install configuration: "debug"
-- Installing: /usr/local/lib64/libcpprealm.so
-- Up-to-date: /usr/local/include/cpprealm/sdk.hpp
-- Up-to-date: /usr/local/include/cpprealm/app.hpp
-- Up-to-date: /usr/local/include/cpprealm/db.hpp
-- Up-to-date: /usr/local/include/cpprealm/notifications.hpp
-- Up-to-date: /usr/local/include/cpprealm/object.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted.hpp
-- Up-to-date: /usr/local/include/cpprealm/results.hpp
-- Up-to-date: /usr/local/include/cpprealm/schema.hpp
-- Up-to-date: /usr/local/include/cpprealm/task.hpp
-- Up-to-date: /usr/local/include/cpprealm/thread_safe_reference.hpp
-- Up-to-date: /usr/local/include/cpprealm/type_info.hpp
-- Up-to-date: /usr/local/include/cpprealm/flex_sync.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_string.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_bool.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_int.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_optional.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_enum.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_timestamp.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_uuid.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_list.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_embedded.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_binary.hpp
-- Up-to-date: /usr/local/include/cpprealm/persisted_mixed.hpp
-- Up-to-date: /usr/local/include/cpprealm/analytics.hpp
-- Up-to-date: /usr/local/include/cpprealm/generic_network_transport.hpp

So build process looks OK.
Then I make project with Realm cpp code
#include <cpprealm/sdk.hpp>

and CMakeList.txt is:
cmake_minimum_required(VERSION 3.5)
project(dbbench LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

add_executable(dbbench main.cpp test_realm.cpp)
target_link_libraries(dbbench fmt cpprealm)

But got error:
dbbench.dir/test_realm.cpp.o -c /home/sergey/Work/dbtest/dbbench/dbbench/test_realm.cpp
In file included from /usr/local/include/cpprealm/sdk.hpp:24,
                 from /home/sergey/Work/dbtest/dbbench/dbbench/test_realm.cpp:16:
/usr/local/include/cpprealm/type_info.hpp:25:10: fatal error: realm/object-store/property.hpp: No such file or directory

I guess some incude files was not installed and add include paths to CMakeLists.txt:
include_directories(SYSTEM "/Work/realm-cpp/realm-core/src/")
include_directories(SYSTEM "/Work/realm-cpp/build.release/realm-core/src/")
include_directories(SYSTEM "/Work/realm-cpp/src/")

With this additions I could compile, but got a lot of link errors:
: && /usr/bin/g++ -DQT_QML_DEBUG -g  CMakeFiles/dbbench.dir/main.cpp.o CMakeFiles/dbbench.dir/test_realm.cpp.o -o dbbench  -Wl,-rpath,/usr/local/lib:  -lfmt  -lcpprealm && :
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dbbench.dir/test_realm.cpp.o: in function `__static_initialization_and_destruction_0(int, int)':
/home/sergey/Work/realm-cpp/src/cpprealm/db.hpp:109: undefined reference to `realm::util::Scheduler::make_default()'
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dbbench.dir/test_realm.cpp.o: in function `realm::util::detail::ExceptionWithBacktraceBase::ExceptionWithBacktraceBase()':
/home/sergey/Work/realm-cpp/realm-core/src/realm/util/backtrace.hpp:101: undefined reference to `realm::util::Backtrace::capture()'
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dbbench.dir/test_realm.cpp.o: in function `realm::util::File::Streambuf::seekpos(std::fpos<__mbstate_t>, std::_Ios_Openmode)':
/home/sergey/Work/realm-cpp/realm-core/src/realm/util/file.hpp:1315: undefined reference to `realm::util::File::seek(long)'
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dbbench.dir/test_realm.cpp.o: in function `realm::util::File::Streambuf::flush()':
/home/sergey/Work/realm-cpp/realm-core/src/realm/util/file.hpp:1323: undefined reference to `realm::util::File::write(char const*, unsigned long)'
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dbbench.dir/test_realm.cpp.o: in function `realm::util::detail::ExceptionWithBacktraceBase::~ExceptionWithBacktraceBase()':
/Work/realm-cpp/realm-core/src/realm/util/backtrace.hpp:98: undefined reference to `realm::util::Backtrace::~Backtrace()'
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dbbench.dir/test_realm.cpp.o: in function `realm::LogicError::LogicError(realm::LogicError::ErrorKind)':
/Work/realm-cpp/realm-core/src/realm/exceptions.hpp:405: undefined reference to `vtable for realm::LogicError'
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /Work/realm-cpp/realm-core/src/realm/exceptions.hpp:405: undefined reference to `vtable for realm::LogicError'

I found several static libs in build directory and one by one added to target_link_libraries, and almost eliminated link errors.
link_directories("/Work/realm-cpp/build.release/realm-core/src/realm/"
    "/Work/realm-cpp/build.release/realm-core/openssl/lib/"
    "/Work/realm-cpp/build.release/realm-core/src/realm/object-store/"
    "/Work/realm-cpp/build.release/realm-core/src/realm/sync/"
)

target_link_libraries(dbbench fmt realm realm-object-store realm-sync uv cpprealm)

but finally got unresolved SSL_write@@OPENSSL_1_1_0
I added ssl & crypto to target_link_libraries and then got a lot of new unresolved:
undefined reference to `realm::make_in_realm_history()'
undefined reference to `realm::Status::Status(realm::ErrorCodes::Error, char const*)'
...

Then I fill tired and guess I do something wrong....
Could you help me to build and test cpp-realm application ?

Can you try with the branch lm/install mentioned here?

This branch is much better - all includes was installed (actually I understand that includes are part of realm-core) and I can compile... But not link - libcpprealm.so was not build, so only static libraries were installed:
sudo cmake --build . --target install | grep [.]a
-- Up-to-date: /usr/local/lib/librealm-dbg.a
-- Up-to-date: /usr/local/lib/librealm-parser-dbg.a
-- Up-to-date: /usr/local/lib/librealm-ffi-static-dbg.a
-- Up-to-date: /usr/local/lib/librealm-object-store-dbg.a
-- Up-to-date: /usr/local/lib64/librealm-sync-dbg.a
sudo cmake --build . --target install | grep [.]so

target_link_libraries(dbbench fmt realm-dbg realm-parser-dbg realm-ffi-static-dbg realm-object-store-dbg realm-sync-dbg uv ssl crypto)
And I have a lot of unresolved:
undefined reference to `realm::realm_analytics::send()'
undefined reference to `realm::make_in_realm_history()'
...

Finally I rearranged libraries in target_link_libraries and built executable.
Unfortunately first simplest action - open database crashed:

Do you have any idea why?
struct MyRealmObject: realm::object {
    realm::persisted<std::string> _id;
    realm::persisted<std::string> typeId;
    realm::persisted<std::vector<std::string>> keys;

    static constexpr auto schema = realm::schema("MyRealmObject",
                                                 realm::property<&MyRealmObject::_id, true>("_id"), // primary key
                                                 realm::property<&MyRealmObject::typeId>("typeId"),
                                                realm::property<&MyRealmObject::keys>("keys"));
};

Very strange, that shouldn't happen. I'll try reproduce on OpenSUSE Tumbleweed.

With your last commit everything works well.
Thank you.


Adding Phi-3 model family as another option in Lepton AI Playground.

Is your feature request related to a problem? Please describe.
Phi-3 is a family of small language models and multi-modal models from Microsoft. Having some Phi-3 models available in Leptonai playground not only enriches the Leptonai ecosystem, but also benefits Phi-3 as more users could be able to access it with ease (thanks to Lepton!).
Describe the solution you'd like
I would like to work on bringing some Phi-3 models to Lepton AI Playground as new options (a new block along side with Llama, Mistral, OpenChat, etc.). The plan is to start with microsoft/Phi-3-vision-128k-instruct.
However I did not find the doc on how to add a new model to the playground. Is there a doc for me to start with? (I am trying to build a photon locally now.)
Please let me know how you think about this idea. I volunteer to do all the required work.
Describe alternatives you've considered
I could also try to build new Phi-3 photon using Lepton framework and share with users in some other way. But I really want to make the model(s) available on Lepton playground to make an easy life for all the users.
Additional context
Phi-3 models are under MIT license. The license information

Currently we do not plan to run Phi-3 as serverless API. You can make a photon and run a dedicated deployment on our platform.

Currently we do not plan to run Phi-3 as serverless API. You can make a photon and run a dedicated deployment on our platform.

Hi, thanks for the kind instruction. This is what I am trying to do now. Let me close the issue for now.


Adopt new control scheme in SliceRenderer

Fix #3035.
This PR introduces PSliceController, a widget that can be used by the both Slice and Contour renderers to position their planes.

@clyne - On it.  Thanks for the catch.


Release 4.8 sync  2021/11/07 - 2021/11/09

This contains important enhancements for registry overrides and using beta version of some resources to allow deployment into IKS environments

/approve

/lgtm


Port over JSON builder avoidance from pydoc

Pydoc gets around this by building the JSON output into Sphinx. Port this over.

The existing logic is:

RTD: https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/projects/tasks.py#L730-L735
Pydoc https://github.com/rtfd/pydoc.io/blob/master/pydoc/sphinx/init.py#L32-L55

The quick version is:

Currently we run an entire second build to output JSON from Sphinx, in order to support search
We should output the JSON as an artifact of the build process
This might hit some edge cases in building people's docs, but it would save around 20-25% of our build resources, which would be a huge win for us

There will likely be some issues with weird encoding and other things, but I'd really like this to get some focus here soon.

@safwanrahman you might find this and the attached PR's interesting!


report dir

please rename to assignment6 as report dir is a final project without programming. assignment 6 is not a final project

It has been changed. Thank you.


Fix hostAliases to support multiple entries

Issue:
Current hostAliases implementation supports only one alias entry.
It doesn't support to add two or more alias entries in /etc/hosts.
Refer: https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/#adding-additional-entries-with-hostaliases
We need flexibility to add multiple alias entries.

This is a breaking change and cannot be introduced like this.
Please bump the chart accordingly, add the artifact hub note and update the documentation so it's clear to anyone updating how to implement this.

@pierluigilenoci bumped the version and added artifact nodes, also updated readme and added example in default values.yaml file.

@saravanan30erd, can you please rebase your PR?

@pierluigilenoci Looks like you already rebased it. If there is anything I need to do?


upload week 4 frontend minutes

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 58.694%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2373

Relevant Lines:
4043

💛  - Coveralls


Code an prompt in the console to ask if user plans to take summer courses

Will not have enough time to implement.


GraphQL APIs/Code to carry workflow YAML to the subscriber (to create on cluster)

closing in favor of https://github.com/litmuschaos/litmus/pull/1856


Content: CPU Scheduling Algorithm - RR

Description
Add the article for RR Scheduling algorithm. Use the following structure:

Introduction
Explanation
Algorithm
Code
Sample I/O
Complexity Analysis

@HarshCasper I would love to work in this Issue Please Assign this to me


performace improvements needed

It is needed to query multiples rows per insertion in order to improve the performance.

Done.... tested with 72274 rows.


Add tests that include tripping validation errors

We recently had a problem where validation errors would cause values not be saved.
We've since released a fix for this, and these tests are confirming the correct behaviour.
It's not doing a lot of code reuse, so I can do some refactoring if needed.
gif

Hey @timarney, take a look at what the new query parameter tests are doing.
For the calendar page in particular, it's not really testing what a user would actually do (we're not getting the checkbox calendar interface, for example), but it is testing what it's possible to do with our app.
If you think it's too weird, I can just go back to clicking the calendar as per usual once we're past the registration page.
How do you feel about it?

Talked about this on slack, but this test isn't actually triggering the "too many days selected" error.
It triggers the "not enough days" error, and then it submits the form, so it only needs 3 days to be selectable.
So I'm going to merge it as-is.


Inaccurate database metrics due to Sharding JDBC plugin

Please answer these questions before submitting your issue.

Why do you submit this issue?
[✓ ] Feature

Feature

Which version of SkyWalking, OS, and JRE?

Skywalking 8.5.0.(latest release)

OS: MacOS Catalina

JRE: openjdk version "1.8.0_265"

What happened?
When we use shrding-jdbc to process jdbc statements, due to sharding-sphere, the original statement will be parsed and breaks up into several actual statements to execute. As we want to obtain metrics like CPM and latency for each original statement, both of them, however, are inaccurate. That's because all the actual statements with high cardinality derived from sharding operation was taken into account when calculating the CPM & Latency, instead of being attributed to a single original statement that we found meaningful.

In my personal view, I believe that the key problem is that the entire branch of sub-trace ranging from /ShardingSphere/JDBCRootInvoke/ endpoint to the last executeQuery span cannot obtain the original jdbc statement. Therefore, tag db.statement enclose the actual statement instead of the original statement, so when aggregating metrics, traffic was dispersed to multiple sharded statements' groups.
To better demonstrate this problem, later I'll upload a small demo of this issue

But it still may be helpful for better understanding the connection between the formal SQL and the actual SQL, since in the real application, the sharding rules can be extremely complex and cause high cardinality, right?
For example, we can use OAL to calculate the metrics (Avg. Latency, Error Rate, etc.) for the formal SQL to understand the performance of SQL execution for some particular business logic. Due to fragmentations of actual SQLs, we are not able to have a general performance view at this level.

But it still may be helpful for better understanding the connection between the formal SQL and the actual SQL, since in the real application, the sharding rules can be extremely complex and cause high cardinality, right?

That is ShardingSphere's community's decision. But, we never calculate metrics for SQL. Where is the requirement from?

For example, we can use OAL to calculate the metrics (Avg. Latency, Error Rate, etc.) for the formal SQL to understand the performance of SQL execution for some particular business logic. Due to fragmentations of actual SQLs, we are not able to have a general performance view at this level.

That is why trace exists, diagnosing performance issue as you described, is not from SQL's level metrics.

But it still may be helpful for better understanding the connection between the formal SQL and the actual SQL, since in the real application, the sharding rules can be extremely complex and cause high cardinality, right?

That is ShardingSphere's community's decision. But, we never calculate metrics for SQL. Where is the requirement from?

In some of our business units, they are using Skywalking and want to have a dashboard for SQL(s) metrics. Due to the high cardinality caused by multi-table JOIN (Cartesian product), it is not easy to check the general performance of SQL in some business logics.

That is why trace exists, diagnosing performance issue as you described, is not from SQL's level metrics.

But traces cannot provide aggregational information like percentiles...

In some of our business units, they are using Skywalking and want to have a dashboard for SQL(s) metrics.

But we don't have anything on upstream, so where could we start to discuss? If this is something not existing in open source, or accepted by the community.

But traces cannot provide aggregational information like percentiles...

Again, no use case for upsteam. We can't discuss a thing not existing in public.

In some of our business units, they are using Skywalking and want to have a dashboard for SQL(s) metrics.

But we don't have anything on upstream, so where could we start to discuss? If this is something not existing in open source, or accepted by the community.

But traces cannot provide aggregational information like percentiles...

Again, no use case for upsteam. We can't discuss a thing not existing in public.

Maybe the original description by the author is somehow confusing.
We do have original SQL (formal one) as a Tag in the Local Span so far, (/ShardingSphere/ParseSQL)

The problem is that the LocalSpan (/ShardingSphere/ParseSQL in the figure) does not represent/carry any real execution information, for example, we cannot get real latency for this span and this also cannot be connected to the "MySQL/JDBC/Statement/executeQuery" span below.
So the only issue we want to discuss here is whether it is possible to duplicate or move the tag (db.statement in /ShardingSphere/ParseSQL) to the /ShardingSphere/executeSQL/ span.
For the OAL and user-defined metrics, users can always write OAL to generate new metrics and add them to the Skywalking dashboard, right?

The problem is that the LocalSpan (/ShardingSphere/ParseSQL in the figure) does not represent/carry any real execution information, for example, we cannot get real latency for this span and this also cannot be connected to the "MySQL/JDBC/Statement/executeQuery" span below.

That is ShardingSphere's plugin issue, it doesn't end in async mode.

So the only issue we want to discuss here is whether it is possible to duplicate or move the tag (db.statement in /ShardingSphere/ParseSQL) to the /ShardingSphere/executeSQL/ span.

What do you mean move? Whether they tag logic SQL statement is pure on plugin implementation level. and nothing related to metrics.

For the OAL and user-defined metrics, users can always write OAL to generate new metrics and add them to the Skywalking dashboard, right?

This description is too general, I have to say yes.

From the naming perspecitve, /ShardingSphere/ParseSQL should not include execution phrase. If you have any doubt, submit a question to ShardingSphere community. The whole plugin was contributed by them, which from SkyWalking's perspective, they are the authority.


Add FINOS Meeting Action to Waltz Project Repo

Description
This pull request adds .github/workflows/meetings.yml to the Waltz project repository for recording and registering Waltz project meeting attendees with FINOS.
This script is managed and maintained by the FINOS team.

Hey @maoo and @TheJuanAndOnly99 👋🏻
It would be awesome if you could review this PR with @davidwatkins73 before it's merged into the Waltz repo.
Thanks!
James.

LGTM!

Hi @davidwatkins73 - Do you mind reviewing the PR and merging when convenient?
Happy Monday!
James 🚀

The action mentions this in the comment block:
# To run this action, you'll need the following secrets defined in https://github.com/finos/<repo name>/settings/secrets :
# - FINOS_TOKEN
# - GIT_CSV_TOKEN

If I try and access: https://github.com/finos/waltz/settings/secrets I get bounced and 404d'.
Does this need to be setup before merging or is it something done at the finos organization level and not at the individual project level ?

The action mentions this in the comment block:
# To run this action, you'll need the following secrets defined in https://github.com/finos/<repo name>/settings/secrets :
# - FINOS_TOKEN
# - GIT_CSV_TOKEN

If I try and access: https://github.com/finos/waltz/settings/secrets I get bounced and 404'd.
Does this need to be setup before merging or is it something done at the finos organization level and not at the individual project level ?

I've already set these secrets, you should be all set.
Let me know how it goes.


Update intro.md

testing commit

Pre-flight checklist

[ ] I have read the Contributing Guidelines on pull requests.
[ ] If this is a code change: I have written unit tests and/or added dogfooding pages to fully verify the new behavior.
[ ] If this is a new API or substantial change: the PR has an accompanying issue (closes #0000) and the maintainers have approved on my working plan.

Motivation

Test Plan

Test links

Deploy preview: https://deploy-preview-_____--docusaurus-2.netlify.app/
Related issues/PRs

@AshokButra this repo is not a place to practice your open source skills
Please stop or you will be banned and reported


Instance: USP Course; Carol Paiva

Cloud Computing Instance Flavor
g3.xl - GPU instance (32 CPUs, 125 GB RAM and A100 GPU)
Description
I'm a PhD student at the Federal University of Juiz de Fora and I'm working on the 3D morphometry of the Amphisbaenia skull. I have several microCT scans and would like to use SlicerMorph to segment and extract models.
ORCID
0000-0002-9398-2159
Email
U2FsdGVkX18ImF5e9Xk7r5JWkEuQmGenCwrYgH5bXXH6NNDP/IFzuorc31CbvJsI
Confirm Email
U2FsdGVkX18ImF5e9Xk7r5JWkEuQmGenCwrYgH5bXXH6NNDP/IFzuorc31CbvJsI
Instance Name
instance-109
The instance name is derived from the GitHub issue number using the template instance-NNN. The instance may also be located in the exosphere interface using this name.
Volume Name
My-Data-109
The volume name is derived from the GitHub issue number using the template My-Data-NNN. The volume may also be located in the exosphere interface using this name.

/create

/unshelve

/unshelved

/unshelved


POP3 getMessage fails with 'Line "<==UUID==>" does not match header format'

We are using zend-mail to fetch emails via POP3 and IMAP.
For integration tests, we run an instance of Flying Rat (https://pypi.org/project/flyingrat/) to mock a mail server with POP3. We build multipart emails in Python using MIMEMultipart and send them to the server via smtplib (all standard code used there, nothing fancy).
Now when I try to fetch the emails in zend-mail via getMessage(), it fails with the following exception::
Line "--===============0544532757501787723==" does not match header format!
vendor/zendframework/zend-mail/src/Storage/Part.php(112):   Zend\Mail\Headers::fromString('MIME-Version: 1...')
vendor/zendframework/zend-mail/src/Storage/Message.php(56): Zend\Mail\Storage\Part->__construct(Array)
vendor/zendframework/zend-mail/src/Storage/Pop3.php(63): Zend\Mail\Storage\Message->__construct(Array)
ourclass.php(298): Zend\Mail\Storage\Pop3->getMessage(1)

I have tried to debug this and printed out the $params  in Message.php _construct.
It looks to me like zend-mail is not able to parse the headers correctly because Flying Rat does not support the TOP command and so the whole message is downloaded. Could this be a bug?
Here is the full debug output of the $params::
 (
     [handler] => Zend\Mail\Storage\Pop3 Object
         (
             [protocol:protected] => Zend\Mail\Protocol\Pop3 Object
                 (
                     [hasTop] =>
                     [socket:protected] => Resource id #9
                     [timestamp:protected] =>
                 )

             [has:protected] => Array
                 (
                     [uniqueid] =>
                     [delete] =>
                     [create] =>
                     [top] =>
                     [fetchPart] =>
                     [flags] =>
                 )

             [iterationPos:protected] => 0
             [iterationMax:protected] =>
             [messageClass:protected] => Zend\Mail\Storage\Message
         )

     [id] => 2
     [headers] => Content-Type: multipart/alternative;
  boundary="===============0544532757501787723=="
 MIME-Version: 1.0
 To<EMAIL_ADDRESS> From<EMAIL_ADDRESS> Subject: Test Url in Text
 Message-Id<EMAIL_ADDRESS>
 --===============0544532757501787723==
 Content-Type: text/plain; charset="us-ascii"
 MIME-Version: 1.0
 Content-Transfer-Encoding: 7bit

 No URL here
 --===============0544532757501787723==
 Content-Type: text/html; charset="us-ascii"
 MIME-Version: 1.0
 Content-Transfer-Encoding: 7bit

 https://example.net
 --===============0544532757501787723==--

     [noToplines] => 1
 )

I have further delved into the source code.
The function public static function fromString($string, $EOL = self::EOL) in Headers.php which parses the full message in this case contains no condition for a break, so it is logical that the exception is thrown when the parser reaches the multipart message body.
I wonder how this bug could go unnoticed until now?

This repository has been closed and moved to laminas/laminas-mail; a new issue has been opened at https://github.com/laminas/laminas-mail/issues/6.


Change Grpc port

When running locally we have multiple functions running and we are using the extension AddDurableTaskClient so we can inject this to start orchestration functions:
services.AddDurableTaskClient(b =>
        {
            b.UseGrpc(grpcConfigure =>
            {
                grpcConfigure.Channel = GrpcChannel.ForAddress($"http://localhost:{PORT}");
            });
        });

How do we change the port so that one function can run GRPC on a different port to another function?  When we try overriding the port we get connection errors from the GRPC framework.
This only affects us locally when developing as we can't have more than one function running at the same time but we would like to work around that limitation.

@RichardBurns1982 - are you using durable task standalone, or durable functions?
There has to be something on the other end of that port listening to this gRPC connection. In Durable Functions, you don't add the client yourself. Instead you is the [DurableClient] DurableTaskClient client input binding to your function - the framework will handle providing a client which talks back to the functions host.
In DurableTask standalone, you need to know what port your gRPC sidecar is running on - but we haven't really made the sidecar support official yet. That is still to come.

@jviau Thanks for the response.
We are using durable functions.
We have multiple function projects, for example:
Function1
Function2
Function3
Each of these we are calling:
services.AddDurableTaskClient(b =>
        {
            b.UseGrpc();
        });

This allows us to take a dependency on DurableTaskClient in our classes from the DI container and this all works really well allowing us to start Orchestration Functions.  As we are not changing any of the GRPC settings it is defaulting to port 4001.  This is all fine if we are running a single function project locally or when deployed to Azure.
However, we have a need when developing locally to run multiple function projects at the same time, all of these are by default listening on port 4001.  What we want to do is change it, when the HostingContext.IsDevelopment() is true and have:
Function1 - Port: 4001
Function2 - Port: 4002
Function3 - Port: 4003
Is this possible?  The reason being we have an integration event framework which has subscriptions in each function project which triggers different orchestration functions in each project.  The integration event handlers are resolved from the DI container with DurableTaskClient  as a dependency.

DurableTaskClient is not meant to be used that way in Durable Functions. The port is supplied from the host extension as part of the input binding process, it is not known at startup.
See documentation here for examples on how to use the [DurableClient] input binding.

Thanks again for the response.
Is there no way to take DurableTaskClient from the ServiceCollection?  Without having to take in the function method using input binding and pass down through methods to our service classes?
We used to do this in .net6 in-process.  Technically this is working fine now except locally when we have multiple functions running locally and they're all trying to run on port 4001.  We've not observed any issues in production, what issues would you expect us to be seeing?
I could provide a sample solution if easier so you can see how we are using this and what alternatives you'd suggest.
Many thanks again, I appreciate the help/responses.

From: Jacob Viau @.>
Sent: Monday, April 17, 2023 5:59:50 PM
To: microsoft/durabletask-dotnet @.>
Cc: Richard Burns @.>; Mention @.>
Subject: Re: [microsoft/durabletask-dotnet] Change Grpc port (Issue #145)
DurableTaskClient is not meant to be used that way in Durable Functions. The port is supplied from the host extension as part of the input binding process, it is not known at startup.
See documentation herehttps://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-isolated-create-first-csharp?pivots=code-editor-vscode#add-functions-to-the-app for examples on how to use the [DurableClient] input binding.
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/durabletask-dotnet/issues/145#issuecomment-1511747340, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQDA45B4VYPO2YXPTIJB7LDXBVZINANCNFSM6AAAAAAW6NAKM4.
You are receiving this because you were mentioned.Message ID: @.***>

For durable functions in dotnet isolated, no there is no way to do this. You simply do not know what the gRPC port to bind to is, it is only supplied to the worker as part of input binding via the [DurableClient] attribute. This isn't to say we won't support to eventually; it totally depends on how/if things change in the functions extensibility model. In-process is different, there is no concept of talking from worker to host there, you are just accessing the backing durable task services directly - so they can be resolved via DI.
When you say "two functions locally", are you talking about two separate function apps entirely, or just two functions within the same project?

We've not observed any issues in production, what issues would you expect us to be seeing?

Want to clarify this. The functions extension code will first attempt to bind to 4001, so it may work for you most of the time. But when that port is reserved it will start trying to get random ports in the 30000 to 31000 range. This is when you will fail to bind to that client.
I understand your desire though, it would be nice to have the client in the DI container. I am currently exploring a feature which may enable this, I'll update this if that happens.

Closing this in favor of #154 - which will solve this issue.

@RichardBurns1982 did you ever find a workaround for this? We are currently experiencing the exact same issue, with getting an error when trying to run two function apps locally both injecting a DurableTaskClient into a service.


clarify some details regarding error objects

fixes https://github.com/json-api/json-api/issues/1496
in json:api 1.1 error objects:

require at least one member be set
clarify that links, if provided, is not required to have BOTH about & type
strengthen status member with a SHOULD

Consider the following response:
{
    "errors": [
        {
            "description": "Some description about the error"
        }
    ]
}

This response is perfectly fine with 1.0, but is now invalid in 1.1. In my opinion, this is a breaking change that belongs to version 2.0 instead of 1.1.

@Art4 Your example document was invalid for v1.0 as well. A description member in errors object is neither allowed in v1.0 not in v1.1.

@jelhan Unknown/undefined properties must be ignored, but are not prohibited. This would be against the "never remove, only add" strategy.
To quote the spec: https://jsonapi.org/format/1.0/#document-structure

Unless otherwise noted, objects defined by this specification MUST NOT contain any additional members. Client and server implementations MUST ignore members not recognized by this specification.


features support

Does it support multiple learning rates and tta?

Hi,

Because u can easily define any model and optimizer by yourself – yes, why not. Just set LRs as u want them to be.

TTAs are also possible, but still in development – I am not kaggle expert, so they have quite low priority. Maybe you can help with this issue.. :)


detect when pthread_rwlock_t is moved

For some implementations of pthreads, the address of pthread_rwlock_t (or its fields) is used to identify the lock. That means that if the contents of a pthread_rwlock_t are moved in memory, effectively a new lock object is created, which is completely independted from the original. Thus we want to detect when when such objects are moved and show an error.
see also #3749 for more context

@rustbot ready

Looks good, thanks. :)
@bors r+

:pushpin: Commit 56fbbcbf1b584ff8bb4a0a673f40ff56988c7968 has been approved by RalfJung
It is now in the queue for this repository.

:hourglass: Testing commit 56fbbcbf1b584ff8bb4a0a673f40ff56988c7968 with merge 1c1524de82505752cd14c6f411de34d405502869...


M2: Support for WASM: Make it easier to bring common workloads to WASM transparently (e.g. Go, Python…

ETA: 2023Q1

what is this @simonwo ? Close?


Add subdirectory value while preparing pyproject.toml if subdirectory key exists

I can confirm the issue. If we out dependency is actually in a subdirectory poetry fails to install it, because dependabot does not add subdirectory into a auto-generate pyproject.yaml

Anyone?

Can you share example dependency manifests that reproduce this problem?

Can you share example dependency manifests that reproduce this problem?

In my case the pyproject.toml file in the project contains the dependency defined in the following form:
[tool.poetry.dependencies]

my_library = {git = "https://github.com/<organisation url>/<repo url>", rev = "v1.2.0", subdirectory = "python"}

The pyproject.yaml file generated by the dependabot does not contain the subdirectory at all, that's why poetry fails to identify the dependency.

I still don't understand the problem well. What is dependabot doing, is it removing the directory key from the pyproject.toml file?
It seems your solution is looking at the directory option in the lockfile, so I guess poetry.lock is also involved? What do you mean by pyproject.yaml, is that a typo for pyproject.toml.
Ideally this would have an accompanying test that shows the problem. But if that's hard (which I understand, our specs are quite tricky), I think creating a small real test repository that shows the problem would help a lot.
A

@deivid-rodriguez no. The thing is that dependabot creates a temp directory, where it places a processed version of pyproject.toml. It reformats it, removes some things and so on and then tries running update command in a venv. That is when things fail, since poetry is unable install a package, because dependabot removed subdirectory from pyproject.toml.

Ok, that's starting to make sense. So the error is a crash while checking for updates, not a bad PR. That points to an error in UpdateChecker. Is that the case? You haven't even copied a backtrace for us, so I assumed this was not an error but a bad PR.
I know it's easier to try to explain the problem, but for us it's really much better to have a sample repository where we can see it. Could you please create that?
By the way, specs seems to be failing somewhere near where this problem should be happening. It may be a flaky failure, but I've never seen it before and it does look suspicious.

@deivid-rodriguez
It is coming from a private org that's why we can't share the exact case 1:1. We were in a big rush trying to fix the thing.
Here us a stacktrace I found on our slack:
updater | 2023/10/05 10:39:00 ERROR <job_731085678> Creating virtualenv app-U0giz7ek-py3.11 in /home/dependabot/.cache/pypoetry/virtualenvs
updater |
updater | Unable to determine package info for path: /home/dependabot/.cache/pypoetry/virtualenvs/app-U0giz7ek-py3.11/src/events
updater |
updater | Command ['/tmp/tmpm9w6jxkp/.venv/bin/python', '-'] errored with the following return code 1
updater |
updater | Output:
updater | Traceback (most recent call last):
updater |   File "<stdin>", line 9, in <module>
updater |   File "/tmp/tmpm9w6jxkp/.venv/lib/python3.11/site-packages/build/__init__.py", line 267, in __init__
updater |     _validate_source_directory(srcdir)
updater |   File "/tmp/tmpm9w6jxkp/.venv/lib/python3.11/site-packages/build/__init__.py", line 135, in _validate_source_directory
updater |     raise BuildException(f'Source {srcdir} does not appear to be a Python project: no pyproject.toml or setup.py')
updater | build.BuildException: Source /home/dependabot/.cache/pypoetry/virtualenvs/app-U0giz7ek-py3.11/src/events does not appear to be a Python project: no pyproject.toml or setup.py
updater |
updater |
updater | Input:
updater | import build
updater | import build.env
updater | import pyproject_hooks
updater |
updater | source = '/home/dependabot/.cache/pypoetry/virtualenvs/app-U0giz7ek-py3.11/src/events'
updater | dest = '/tmp/tmpm9w6jxkp/dist'
updater |
updater | with build.env.IsolatedEnvBuilder() as env:
updater |     builder = build.ProjectBuilder(
updater |         srcdir=source,
updater |         scripts_dir=env.scripts_dir,
updater |         python_executable=env.executable,
updater |         runner=pyproject_hooks.quiet_subprocess_runner,
updater |     )
updater |     env.install(builder.build_system_requires)
updater |     env.install(builder.get_requires_for_build('wheel'))
updater |     builder.metadata_path(dest)
updater |
updater | No fallback setup.py file was found to generate egg_info.

It does not give you a stacktrace per se, but since you already have a PR.
As you can see it tries to install a package at src/events however it should be src/events/python.
We command it fails on is:
pyenv exec poetry update pytz --lock --no-interaction

Ok!
I'm going to go through some common Python errors we see in our error tracker next week, so I'll probably find this particular issue happening on a public repo. I'll use that chance to be able to reproduce this problem and review this PR and maybe add a test to it.

Just to add more context https://github.com/dependabot/dependabot-core/blob/main/python/lib/dependabot/python/helpers.rb#L16
This is where I added a debugger statement.
I downloaded dependabot docker image, dropped into cli there and:

Ran dependabot update
Kept skipping breakpoint till command became pyenv exec poetry update pytz --lock --no-interaction (the one I saw failing in "stacktrace")
I ran Open3.capture3("pwd") to see what is the working directory it tries to execute poetry command in
This gave a temp folder dependabot created, where it placed processed pyproject file
I manually went to that folder, ran pyenv exec poetry update pytz --lock --no-interaction
It gave a failure. I started investigating pyproject.toml file which was there. Was surprised to see that it looks differently from our original one. Started investigating it and found out that the main difference is the absence of subdirectory

@deivid-rodriguez
Good day. By chance any news on this PR?

No, not yet. Just reiterate that a public repository that illustrates the bug being fixed here will speed things up.

@deivid-rodriguez
A friendly bump.
For your convenience I prepared a testing repo which shows the problem.
Imaginary service which imports a library:
https://github.com/sysradium/dependabot-service/
A library with a subdirectory:
https://github.com/sysradium/dependabot-subdir-lib
To see the problem:
> gcl git@github.com:sysradium/dependabot-service.git

> cd ./dependabot-service

> dependabot update pip sysradium/dependabot-service --commit main

    cli | 2023/10/24 13:00:38 Adding missing credentials-metadata into job definition
    cli | 2023/10/24 13:00:38 using image ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest at sha256:63dc6cee93db02fb70e39e7278f4a90fd0b39376572f9813032dfa89672014e1
    cli | 2023/10/24 13:00:38 using image ghcr.io/dependabot/dependabot-updater-pip at sha256:a16af7aba8898b8148f6f5709bd64399ca19f043bd9793a77953277bf774feeb
  proxy | 2023/10/24 13:00:38 proxy starting, commit: fd948d06eb7d332426f2a1577cd4f69a0d7cf522
  proxy | 2023/10/24 13:00:38 initializing metrics client: No address passed and autodetection from environment failed
  proxy | 2023/10/24 13:00:38 GitHubAPIHandler has no app access tokens
updater | Updating certificates in /etc/ssl/certs...
  proxy | 2023/10/24 13:00:38 Listening (:1080)
updater | rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
updater | 1 added, 0 removed; done.
updater | Running hooks in /etc/ca-certificates/update.d...
updater | done.
updater | 2023/10/24 13:00:39 INFO Raven 3.1.2 configured not to capture errors: DSN not set
updater | 2023/10/24 13:00:40 INFO Starting job processing
  proxy | 2023/10/24 13:00:40 [002] GET https://api.github.com:443/repos/sysradium/dependabot-service/contents/?ref=main
  proxy | 2023/10/24 13:00:40 [002] 200 https://api.github.com:443/repos/sysradium/dependabot-service/contents/?ref=main
  proxy | 2023/10/24 13:00:40 [004] GET https://api.github.com:443/repos/sysradium/dependabot-service/contents/pyproject.toml?ref=main
  proxy | 2023/10/24 13:00:40 [004] 200 https://api.github.com:443/repos/sysradium/dependabot-service/contents/pyproject.toml?ref=main
  proxy | 2023/10/24 13:00:40 [006] GET https://api.github.com:443/repos/sysradium/dependabot-service/contents/poetry.lock?ref=main
  proxy | 2023/10/24 13:00:41 [006] 200 https://api.github.com:443/repos/sysradium/dependabot-service/contents/poetry.lock?ref=main
updater | 2023/10/24 13:00:41 INFO Finished job processing
updater | 2023/10/24 13:00:41 INFO Raven 3.1.2 configured not to capture errors: DSN not set
updater | 2023/10/24 13:00:41 INFO Starting job processing
  proxy | 2023/10/24 13:00:43 [007] POST http://host.docker.internal:60804/update_jobs/cli/update_dependency_list
{"data":{"dependencies":[{"name":"jinja2","requirements":[],"version":"3.0.3"},{"name":"markupsafe","requirements":[],"version":"2.1.3"},{"name":"protobuf","requirements":[],"version":"4.23.2"},{"name":"validate-email","requirements":[],"version":"1.3"}],"dependency_files":["/pyproject.toml","/poetry.lock"]},"type":"update_dependency_list"}
  proxy | 2023/10/24 13:00:43 [007] 200 http://host.docker.internal:60804/update_jobs/cli/update_dependency_list
  proxy | 2023/10/24 13:00:43 [008] POST http://host.docker.internal:60804/update_jobs/cli/increment_metric
{"data":{"metric":"updater.started","tags":{"operation":"update_all_versions"}},"type":"increment_metric"}
  proxy | 2023/10/24 13:00:43 [008] 200 http://host.docker.internal:60804/update_jobs/cli/increment_metric
updater | 2023/10/24 13:00:43 INFO Starting update job for sysradium/dependabot-service
updater | 2023/10/24 13:00:43 INFO Checking all dependencies for version updates...
updater | 2023/10/24 13:00:43 INFO Checking if jinja2 3.0.3 needs updating
  proxy | 2023/10/24 13:00:43 [010] GET https://pypi.org:443/simple/jinja2/
  proxy | 2023/10/24 13:00:43 [010] 200 https://pypi.org:443/simple/jinja2/
updater | 2023/10/24 13:00:44 INFO Latest version is 3.1.2
  proxy | 2023/10/24 13:00:45 [012] GET https://github.com:443/sysradium/dependabot-subdir-lib.git/info/refs?service=git-upload-pack
  proxy | 2023/10/24 13:00:45 [012] 200 https://github.com:443/sysradium/dependabot-subdir-lib.git/info/refs?service=git-upload-pack
  proxy | 2023/10/24 13:00:45 [014] POST https://github.com:443/sysradium/dependabot-subdir-lib.git/git-upload-pack
  proxy | 2023/10/24 13:00:45 [014] 200 https://github.com:443/sysradium/dependabot-subdir-lib.git/git-upload-pack
  proxy | 2023/10/24 13:00:45 [016] POST https://github.com:443/sysradium/dependabot-subdir-lib.git/git-upload-pack
  proxy | 2023/10/24 13:00:45 [016] 200 https://github.com:443/sysradium/dependabot-subdir-lib.git/git-upload-pack
  proxy | 2023/10/24 13:00:47 [018] GET https://pypi.org:443/simple/build/
  proxy | 2023/10/24 13:00:47 [018] 200 https://pypi.org:443/simple/build/
  proxy | 2023/10/24 13:00:47 [020] GET https://files.pythonhosted.org:443/packages/58/91/17b00d5fac63d3dca605f1b8269ba3c65e98059e1fd99d00283e42a454f0/build-0.10.0-py3-none-any.whl
  proxy | 2023/10/24 13:00:47 [020] 200 https://files.pythonhosted.org:443/packages/58/91/17b00d5fac63d3dca605f1b8269ba3c65e98059e1fd99d00283e42a454f0/build-0.10.0-py3-none-any.whl
  proxy | 2023/10/24 13:00:47 [022] GET https://pypi.org:443/simple/pyproject-hooks/
  proxy | 2023/10/24 13:00:47 [022] 200 https://pypi.org:443/simple/pyproject-hooks/
  proxy | 2023/10/24 13:00:47 [024] GET https://files.pythonhosted.org:443/packages/d5/ea/9ae603de7fbb3df820b23a70f6aff92bf8c7770043254ad8d2dc9d6bcba4/pyproject_hooks-1.0.0-py3-none-any.whl
  proxy | 2023/10/24 13:00:47 [024] 200 https://files.pythonhosted.org:443/packages/d5/ea/9ae603de7fbb3df820b23a70f6aff92bf8c7770043254ad8d2dc9d6bcba4/pyproject_hooks-1.0.0-py3-none-any.whl
  proxy | 2023/10/24 13:00:47 [026] GET https://pypi.org:443/simple/packaging/
  proxy | 2023/10/24 13:00:48 [026] 200 https://pypi.org:443/simple/packaging/
  proxy | 2023/10/24 13:00:48 [028] GET https://files.pythonhosted.org:443/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
  proxy | 2023/10/24 13:00:48 [028] 200 https://files.pythonhosted.org:443/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
  proxy | 2023/10/24 13:00:48 [030] GET https://files.pythonhosted.org:443/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl
  proxy | 2023/10/24 13:00:48 [030] 200 https://files.pythonhosted.org:443/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl
  proxy | 2023/10/24 13:00:50 [031] POST http://host.docker.internal:60804/update_jobs/cli/record_update_job_error
{"data":{"error-type":"unknown_error","error-details":null},"type":"record_update_job_error"}
  proxy | 2023/10/24 13:00:50 [031] 200 http://host.docker.internal:60804/update_jobs/cli/record_update_job_error
updater | 2023/10/24 13:00:50 ERROR Error processing jinja2 (Dependabot::SharedHelpers::HelperSubprocessFailed)
updater | 2023/10/24 13:00:50 ERROR Creating virtualenv dependabot-service-uzoQjMJP-py3.11 in /home/dependabot/.cache/pypoetry/virtualenvs
updater |
updater | Unable to determine package info for path: /home/dependabot/.cache/pypoetry/virtualenvs/dependabot-service-uzoQjMJP-py3.11/src/dependabot-subdir-lib
updater |
updater | Command ['/tmp/tmp24sdv0ke/.venv/bin/python', '-'] errored with the following return code 1
updater |
updater | Output:
updater | Traceback (most recent call last):
updater |   File "<stdin>", line 9, in <module>
updater |   File "/tmp/tmp24sdv0ke/.venv/lib/python3.11/site-packages/build/__init__.py", line 267, in __init__
updater |     _validate_source_directory(srcdir)
updater |   File "/tmp/tmp24sdv0ke/.venv/lib/python3.11/site-packages/build/__init__.py", line 135, in _validate_source_directory
updater |     raise BuildException(f'Source {srcdir} does not appear to be a Python project: no pyproject.toml or setup.py')
updater | build.BuildException: Source /home/dependabot/.cache/pypoetry/virtualenvs/dependabot-service-uzoQjMJP-py3.11/src/dependabot-subdir-lib does not appear to be a Python project: no pyproject.toml or setup.py
updater |
updater |
updater | Input:
updater | import build
updater | import build.env
updater | import pyproject_hooks
updater |
updater | source = '/home/dependabot/.cache/pypoetry/virtualenvs/dependabot-service-uzoQjMJP-py3.11/src/dependabot-subdir-lib'
updater | dest = '/tmp/tmp24sdv0ke/dist'
updater |
updater | with build.env.IsolatedEnvBuilder() as env:
updater |     builder = build.ProjectBuilder(
updater |         srcdir=source,
updater |         scripts_dir=env.scripts_dir,
updater |         python_executable=env.executable,
updater |         runner=pyproject_hooks.quiet_subprocess_runner,
updater |     )
updater |     env.install(builder.build_system_requires)
updater |     env.install(builder.get_requires_for_build('wheel'))
updater |     builder.metadata_path(dest)
updater |
updater | No fallback setup.py file was found to generate egg_info.
updater | 2023/10/24 13:00:50 ERROR /home/dependabot/python/lib/dependabot/python/helpers.rb:23:in `run_poetry_command'
updater | 2023/10/24 13:00:50 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/poetry_version_resolver.rb:312:in `run_poetry_command'
updater | 2023/10/24 13:00:50 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/poetry_version_resolver.rb:150:in `run_poetry_update_command'
u
....

Thanks! That's really helpful, I can repro this now. Let me have a look!

Fix looks great, I just added a test using the example files you provided!

@deivid-rodriguez let us merge then?

I will try to deploy & merge today. Just a little bit more patience, please.

@deivid-rodriguez no problem. Just wanted to know if there is anything we should do to help you.
OK, than just waiting.
Thank you.

Thanks for your patience, this is now live!


INTERACTION_ALREADY_REPLIED Error

When using CommandInteraction, the INTERACTION_ALREADY_REPLIED error can sometimes show.

(Image Provided by "63poopy" on Discord)

Should be fixed in v3.2.2. This issue shall reopen if this problem still occurs.


Support quick outline for PHP?

Is that possible to add a function list panel?
Besides, the "Go to Symbol" function seems not working for php file. I am working with vscode v0.10.6

This is currently not implemented but a valid feature request.

This is duplicate of #132 I think also you should vote this: https://visualstudio.uservoice.com/forums/293070-visual-studio-code/suggestions/7752606-php-intellisense
PHP IntelliSense is required for symbol navigation (e.g. symbols, Go to definition, ...)

Closing as a duplicate of #132, thanks @Ciantic


No Preservation Lock for Retention Policies for Teams?

[Enter feedback here]
Hi team ... can you please confirm if retention policies for Teams can have preservation lock enabled? From the screenshot, it looks like the answer is no since the Edit option is greyed out. Thanks - M.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f55c52a5-5841-b9c9-1607-2013dc3f2e20
Version Independent ID: 16d93458-5e01-646f-dac5-31306cc7f3fb
Content: Retention policies in Microsoft Teams - Microsoft Teams
Content Source: Teams/retention-policies.md
Service: msteams
GitHub Login: @cabailey
Microsoft Alias: cabailey

@Michael-Sampson Thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.

@Michael-Sampson  No need to open a service ticket - I can confirm that a retention policy for Teams supports Preservation Lock and you'll find supporting documentation here: https://docs.microsoft.com/en-us/microsoft-365/compliance/create-retention-policies?view=o365-worldwide#lock-a-retention-policy-by-using-powershell


easier way to get exception reason out of json_sax_dom_callback_parser without exceptions

Describe the feature in as much detail as possible.

I am forced to disable exceptions in my project as I am targeting WASM with Emscripten.
I was able to use the code below to parse json from a std::string without using any exceptions.  This works great for me however if I do hit a parse error, I would like to know what the error was.  Looking at class parser::parse() it uses a json_sax_dom_callback_parser, which inturn calls parse_error().  But there is no way to get the details of what happened from there when allow_exceptions is false.  Is it possible to add a callback somewhere that accepts a parse_error or something like that?

Include sample usage where appropriate.

This is the function I'm using so far:
using nlohmann::detail::input_buffer_adapter ;
std::optional<nlohmann::json> jsonParseExceptionFree(const std::string& str) {
    const char* first = str.c_str();
    auto len = str.size();

    // The resulting json object
    nlohmann::json j;

    // some stuff I don't understand
    auto ia = std::make_shared<input_buffer_adapter>(reinterpret_cast<const char*>(&(*first)), len);
    nlohmann::detail::parser<nlohmann::basic_json<>>(ia, nullptr, false).parse(false, j);

    if( j.is_discarded() ) {
        // parse error
        return {};
    } else {
        // parsed correctly
        return j;
    }
}

Thanks for the help, I really love this library but the internals are more complicated than I understand.

You can have it a bit simpler:
#include <iostream>
#include "json.hpp"

using json = nlohmann::json;

json jsonParseExceptionFree(const std::string& s)
{
    auto j = json::parse(s, nullptr, false);
    return j.is_discarded() ? json() : j;
}

int main() {
    std::string s = "{}}";
    auto js = jsonParseExceptionFree(s);
    std::cout << js << std::endl;
}

Oh, now I see the actual issue. If you want more access to diagnostic information, you may need to dive into the SAX interface. You may want to have a look at the json_sax_dom_parser (https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp#L145) parser. The you can then access the information where the exceptions are created from.

Ok.  I'll take a look at overriding just the parse_error as the rest of it works perfectly.  Thanks!


PR: Add basic config dlg tests for plugins

Description of Changes

[x] Wrote at least one-line docstrings (for any new functions)
[x] Added unit test(s) covering the changes (if testable)

[ ] Included a screenshot or animation (if affecting the UI, see Licecap)

In preparing for the changes that will be made on the config system, the plugins now have a basic fixture to test the config dialog. These tests also allow to show what are the methods in the main window that should be exposed by the api (see the mainwindowmocks)
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @goanpeca

Hello @goanpeca! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file spyder/plugins/breakpoints/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/console/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/editor/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/explorer/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/findinfiles/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/history/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/ipythonconsole/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/languageserver/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/onlinehelp/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/outlineexplorer/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/plots/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/profiler/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/pylint/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/variableexplorer/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/plugins/workingdirectory/tests/test_config_dialog.py:

Line 4:2: W291 trailing whitespace

In the file spyder/preferences/tests/conftest.py:

Line 46:52: E221 multiple spaces before operator

To explain a little better what is going on. This tests is using parametrizable fixtures which can be achieved by using the indirect option. Unfortunately pytest explanation for this usage is a bit cryptic.
Beside the pytest "weirdness" the tests in here allow the following:

Use a single fixture to test any type of config plugin. This will be also usable by any plugin developers
Test the plugins config pages in standalone mode (that is why we use windows mocks on some places)
Test the save_to_conf method on all the plugin config pages. This is to check that something like what happened on this issue (https://github.com/spyder-ide/spyder/pull/9706), does not happen with the plugins.

Remove test for plugins with no config page @ccordoba12


Possible bug in Element.documentOffset

https://dartpad.dartlang.org/a02bb87e98ca42dbb086cc4ce988d2bb
Explanation: The little purple box is supposed to be in the upper-left corner of the green element because I am giving it the 'documentOffset.x and documentOffset.y' properties.
Somehow it won't go there, I've posted a stackoverflow question which advised to report it as a bug.

See also http://stackoverflow.com/questions/39333791/element-offset-element-documentoffset-resulten-in-false-values

Judging by the SO post and the code, this is was intended to be a Dart SDK bug report. Closing here; please reopen in the SDK issues tracker (https://github.com/dart-lang/sdk/issues) if it's still a problem.


puppet: Restrict postfix incoming addresses to postmaster and zulip.

This removes the possibility of local user enumeration via RCPT TO.
Testing Plan: Applied on a test voyager instance.  Before:
220 alexmv-prod.zulipdev.org ESMTP Postfix (Zulip)
HELO localhost
250 alexmv-prod.zulipdev.org
MAIL FROM<EMAIL_ADDRESS>250 2.1.0 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>550 5.1.1<EMAIL_ADDRESS>Recipient address rejected: User unknown in local recipient table
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok

After:
220 alexmv-prod.zulipdev.org ESMTP Postfix (Zulip)
HELO localhost
250 alexmv-prod.zulipdev.org
MAIL FROM<EMAIL_ADDRESS>250 2.1.0 Ok
RCPT TO<EMAIL_ADDRESS>554 5.7.1<EMAIL_ADDRESS>Recipient address rejected: Access denied
RCPT TO<EMAIL_ADDRESS>554 5.7.1<EMAIL_ADDRESS>Recipient address rejected: Access denied
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>554 5.7.1<EMAIL_ADDRESS>Recipient address rejected: Access denied
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok
RCPT TO<EMAIL_ADDRESS>250 2.1.5 Ok

lgtm, feel free to merge to master and 3.x after CI passes.


Fix checks for missing parameters

@jchodera - I was under the impression from what you wrote describing your initial code that an exception would be raised if parameters were missing for any of the major required force types in a molecule (i.e. bonds, angles, torsions, LJ), but this seems not to be the case in general. I see this being done for bonds (https://github.com/open-forcefield-group/smarty/blob/master/smarty/forcefield.py#L1234) but for none of the other force terms.
I happened across this because I was starting with a specific SMIRFF and trying to make it more general, and I found that I had certain angle terms which simply HAD no angle parameters.
We'll either need to (ASAP) implement this checking, or make our basic SMIRFFs assign generic dummy parameters to all force terms and then check that these get overridden.

Not a problem---I can add this right now.
I think I was hoping to get topological angle and torsion enumeration into simtk.openmm.app.Topology first, but then that dropped by the wayside. I'll add that to our _Topology class for now, and we can migrate those routines to OpenMM later.
Important question: What (if anything) do you want to do about checking for impropers?

@jchodera - don't check impropers. I don't think we have any way at this point to tell when they should/shouldn't be applied. That will have to be revisited in a separate issue which will involve a lot more chemistry.

OK, nearly done.

Any suggestions for tests? Did you say you have an example of a deficient forcefield file that is missing some terms?

Tests -- easiest would be to just take one of the existing sample forcefields under tests and remove some of the most generic parameters. Give me a moment and I'll give you something concrete.

So, I'd propose you add a really simple test where you attempt to create a molecule from SMILES (say, CCCC or something similar) and make a dummy forcefield (like those in the top of test_forcefield.py) which is missing required parameters -- so, for example, use the initial "ffxml_contents" there but remove the first two angle parameters (a0001 and a0002), the generic torsion t0001, the torsions t0004 and t0005, and the nonbonded term for 6X4, n0008. Then check that the right exceptions get raised. Those removals should break angles, proper torsions, and nonbonded.

This seems more complex than the original code changes. Do you mind if I open a PR and have you implement the tests? Happy to fix if tests fail, but it sounds like it would be more efficient for you to put in the tests.

I could do that. It's not complicated at all though -- just pick some arbitrary simple molecule and delete some of the parameters it uses.

Another question: What about i-j-k-i torsions for three-membered rings? Those should be excluded from torsions, right?

Sorry, @jchodera - what do you mean by "excluded from torsions"? Do you mean that they don't count as a normal torsion? Or are you asking whether we should check whether they are assigned?
I believe they are valid torsions, but maybe I'm missing something.

I believe they are valid torsions, but maybe I'm missing something.

I don't think they can be---they can't ever be anything but 0 degrees.

I agree they are not useful. But they are assigned by existing AMBER-style force fields, I believe.
(Specifically, if we apply the AlkEthOH ffxml to three-membered rings, we get exactly the same force terms as parm@frosst, right now, so unless we're NOT currently assigning these, it means that AMBER-style force fields ARE.)

I agree they are not useful. But they are assigned by existing AMBER-style force fields, I believe.

Haha. I have to run off in a minute but I can make a note to check tomorrow -- I'd just have to use the labeler on a sample molecule to see if we're assigning a torsion to those in AlkEthOH. Whatever we're doing exactly matches what AMBER is doing, so it's an easy check.

 Check out the pucker in the middle ring.

Another question: What about i-j-k-i torsions for three-membered rings? Those should be excluded from torsions, right?

OK, @jchodera - looks like my earlier comments were mistaken. For i-j-k-i torsions for three-membered rings, neither us nor AMBER are currently assigning these. So we'll need to NOT look for parameters being assigned for those.

Got it. We now exclude checks for i-j-k-i torsions in the PR I created.


Screenshots in Readme.md File

Improve readme and documentation using Screenshots

(Love the Changelogs.)

Thanks for opening this issue!
A screenshot of the index page or from a request might be nice :)

This issue was closed by 132421916ec3fd00bc03c7a51f8eb3e0d0788bd1

Glad to see that it was added awesome work!


SUMMARY
#10427 broke saving queries in sql lab with the following error:
/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py:2505: SAWarning: Usage of the 'Session.add()' operation is not currently supported within the execution stage of the flush process. Results may not be consistent.  Consider using alternative event listeners or connection-level operations instead.
  "event listeners or connection-level operations instead." % method
ERROR:flask_appbuilder.models.sqla.interface:Add record error: This transaction is closed
Traceback (most recent call last):
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2597, in _flush
    flush_context.execute()
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
    rec.execute(self)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 624, in execute
    uow,
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 326, in delete_obj
    _organize_states_for_delete(base_mapper, states, uowtransaction)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 463, in _organize_states_for_delete
    base_mapper, uowtransaction, states
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1602, in _connections_for_states
    connection = uowtransaction.transaction.connection(base_mapper)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 311, in connection
    self._assert_active()
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 304, in _assert_active
    raise sa_exc.ResourceClosedError(closed_msg)
sqlalchemy.exc.ResourceClosedError: This transaction is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/flask_appbuilder/models/sqla/interface.py", line 585, in add
    self.session.commit()
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1036, in commit
    self.transaction.commit()
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 503, in commit
    self._prepare_impl()
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 482, in _prepare_impl
    self.session.flush()
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2496, in flush
    self._flush(objects)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2637, in _flush
    transaction.rollback(_capture_exception=True)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 81, in __exit__
    compat.raise_(value, with_traceback=traceback)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
    raise exception
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2637, in _flush
    transaction.rollback(_capture_exception=True)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 519, in rollback
    self._assert_active(prepared_ok=True, rollback_ok=True)
  File "/Users/erik_ritter/repos/github.com/incubator-superset/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 304, in _assert_active
    raise sa_exc.ResourceClosedError(closed_msg)
sqlalchemy.exc.ResourceClosedError: This transaction is closed

This reverts the PR and attempts to fix the merge conflicts on top of it as I couldn't find any specific line at fault here
TEST PLAN
CI, saving queries work
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

to: @john-bodley @bkyryliuk @dpgaspar @villebro

I can't help but wonder if it's the scoped sessions at play.


None of the examples work on Firefox

I see that #4 is closed but it still doesn't work in Firefox.

Yes, I see that the error is happening again. let me see

Cool, working now!


Droplet improvements

Make it easier to regenerate the DigitalOcean base image, and give it a little more flexibility (IPv6 addresses, selectable regions so it can be closer to the user).
Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

I ended up doing a review; lgtm.


Add support for accelerated arrow exchange with python

This is the first step in adding in support for accelerating the data exchange between spark and python processes.
It has some issues currently that I plan to address in a follow on PR, but fixing some of them will require changes to CUDF, which could only go into the 0.16 release or later.

We don't release the GPU semaphore after sending data to the python process, and we don't acquire it when we get data back. This results in bad performance for python processes that take a long time to run.  It requires changes to cudf to be able to fix this.
We have a size limit on some batches read from the python process.  This requires changes to cudf to be able to fix.  Part of the issue is that larger batches tend to be much more efficient so this is preventing testing to see what is the ideal batch size.
Need tests for String and other data types besides int types.
Need documentation about how to tune this for ideal performance, and call out https://issues.apache.org/jira/browse/SPARK-32612
Need to address TODOs in the code to make it so that cached data can spill from GPU memory if needed.  This is really a requirement if we release the GPU semaphore because for now we will not go much past a single batch.
Need to do profiling, especially around the split code to see what is the ideal way to split a large incoming batch up.

This partially addresses #305 but not enough to close it.

build

build

All of this got merged into #640 so closing this as a duplicate for now.


2.x Refactorings/reactive registry

NetflixOSS » eureka » eureka-pull-requests #263 SUCCESS
This pull request looks good

Looks good


feat(iOS): Image loading success or failure callback

Q
A

Fixed Issues?
Fixes #1, Fixes #2

Patch: Bug Fix?

Major: Breaking Change?

Minor: New Feature?
Yes

Tests Added + Pass?
Yes

later


.NET MAUI: use 8.0.20 package version

Updates the package versions from 8.0.6 -> 8.0.20
https://github.com/dotnet/maui/releases/tag/8.0.20

cc @cgutierrez-gis


Ruby: Consider changing smoketest output path

It may be better to have ruby smoketests generate to the 'smoke_test/' path. This issue is to start a discussion to weigh the pros and cons of changing the path.

FYI right now in Java we put all tests under the same path. Maybe worth considering changing the smoke test path across languages.

@geigerj I think your work to put the smoke tests in the acceptance test scope makes this obsolete. Closing. Reopen if I am incorrect.


4.1.0-alpha2 The prefix "tools" for attribute "tools:ignore" associated with an element type "uses-permission" is not bound.

Describe the bug
Build error when upgrading to 4.1.0-alpha2:
Execution failed for task ':app:generateReleaseBuildConfig'.
> Failed to calculate the value of task ':app:generateReleaseBuildConfig' property 'buildConfigPackageName'.
   > Failed to query the value of property 'packageName'.
      > org.xml.sax.SAXParseException; systemId: file:/home/lasse/Documents/viperdev/runtogether/runtogether/android/app/src/main/AndroidManifest.xml; lineNumber: 15; columnNumber: 122; The prefix "tools" for attribute "tools:ignore" associated with an element type "uses-permission" is not bound.

To Reproduce
Add alpha2 to existing project
Change the config to the new context object
flutter clean; flutter build apk --release
Expected behavior
Build should pass.
It did when I removed that line from the android xml FWIW but your docs explicitly tell me to add it. Maybe this is just a docs issue, I don't know that xml file that well but felt you'd like to get this feedback.
Desktop (please complete the following information):

OS: Ubuntu

Smartphone (please complete the following information):
Not relevant since it doesn't build.

maybe you can refer to https://stackoverflow.com/questions/38072616/permission-required-to-use-usagestatsmanager

Yep, I recommend adding that to the readme for the next release to make that issue not appear for other users. I'm fine personally.

If you still face issues, please try upgrading to the new version to see if it works for you.


Updated upgrades from FFG squadbuilder

Imports the following from FFG's squadbuilder for upgrade cards:

Point costs (though it looks like no changes were require)
Image urls
Artwork urls (new!)
Unique IDs ffg_id (new!)

Like with @guidokessels, I used Prettier to reformat after running my script, so there will be some formatting changes.  The image and artwork was added to the appropriate sides.
Some manual verification should be done to make sure I didn't make any errors mapping the FFG data to our data.
Things to verify:

Image shows the correct upgrade
Artwork shows the correct upgrade
ffg_id is correct (can be checked here: https://squadbuilder.fantasyflightgames.com/api/cards/, though if it matches the ID in the image URLs, it will be correct)

This is great!
I was in the middle of doing these changes but this saves me a lot of time. Thanks :)
Did you use a script to generate this? If so, could you share your script? I can merge it with my script to update the pilot data and I can run the scripts every time FFG updates their app to get the new data.

FFG seems to treat dual-sided cards as two different cards, and each have their own ffg_id. We'll have to move that property into sides as well.

My script was a couple nodejs functions in another project that I was working on ... happy to share that but not sure if that's compatible with your script.

@tprovick If you could share what you have I can see if I can use it. It'd be great to be able to run just a single command and update everything :)
What're you building btw?

@guidokessels I stream X-Wing and other FFG games and created an overlay manager with NodeCG that allows me to live-update ships, do card animations and other stuff.  Our channel is here: https://youtube.com/vttvlive  I didn't make the first-gen of the tool available publicly, but I'm rebuilding the second gen and I'm open-sourcing it.
I'll move my script to its own project and try to share that today.

@tprovick Thanks!