Calendar ID is not a number for DeleteEvent

Whenever I execute deviceCalendarPlugin.deleteEvent, I get the following error

[599] Device calendar plugin ran into an issue. Platform specific exception [400], with message :"Calendar ID is not a number", has been thrown.

The code I execute is below.
var permissionsGranted = await _requestCalendarPermissions();

if (!permissionsGranted) {
   return null;
}

//Attempt to delete the event
var result = await _deviceCalendarPlugin.deleteEvent('4', 'eventId');

Can you try following the example where it retrieves the calendar and uses the id of the calendar in the deleteEvent method?

@nickrandolph that's exactly what I did. The '4' as the calendar ID in my example was just for brevity.
Actually, when I execute the code to retrieve the calendar ID, that's the ID I get. The ID works to create the event just fine, but fails to delete it with that error.

@elyctom which platform is this on? can you try the other platform (eg if you've been working on iOS can you check to see if the issue is there on Android, and vice versa)

@nickrandolph, this is happening on Android. I'm having some pod install issues on iOS now, so as soon as I get that build working I'll confirm the iOS behavior.

i got this error
PlatformException(404, The calendar with the ID 1110 could not be found, null)

any idea how to solve this?

@nickrandolph it turns out the fault was all my own. Here's what really happened.

When creating the event, I actually specified an event ID (not knowing that the platform/plugin was going to generate it's own ID for the event and return that ID as a part of the event creation result;
When I was performing the subsequent deletion, I then used the event ID I had specified, instead of the one returned from the creation result

When I used the event ID from the creation result, it worked just fine. The error provided in the PlatformException is however misleading, since the problem wasn't with the calendar ID, but with the event ID.

var currentLocation =
getLocation(await FlutterNativeTimezone.getLocalTimezone());
setLocalLocation(currentLocation);
print(currentLocation);
await Future.delayed(const Duration(seconds: 2));
var eventToCreate = Event(
  "1",
  //  eventId: "50",
  title: "calendarEventModel 12345",
  description: "eventDescription",
  start: TZDateTime.now(currentLocation),
  end: TZDateTime.now(currentLocation).add(Duration(hours: 10)),
);

final createEventResult =
    await deviceCalendarPlugin.createOrUpdateEvent(eventToCreate);
if (createEventResult!.isSuccess) {
  Utils.showTopMessage("Event added to Calendar", Colors.green);
  Get.back();
}

}
i used  following packages
device_calendar: ^4.3.0
flutter_native_timezone:
this is working perfectly in my app.
events are created sucessfully and display into my device calendar.

@Inihood1 please file a new issue with more details.

var currentLocation = getLocation(await FlutterNativeTimezone.getLocalTimezone()); setLocalLocation(currentLocation); print(currentLocation);
await Future.delayed(const Duration(seconds: 2));
var eventToCreate = Event(
  "1",
  //  eventId: "50",
  title: "calendarEventModel 12345",
  description: "eventDescription",
  start: TZDateTime.now(currentLocation),
  end: TZDateTime.now(currentLocation).add(Duration(hours: 10)),
);

final createEventResult =
    await deviceCalendarPlugin.createOrUpdateEvent(eventToCreate);
if (createEventResult!.isSuccess) {
  Utils.showTopMessage("Event added to Calendar", Colors.green);
  Get.back();
}

}
i used following packages device_calendar: ^4.3.0 flutter_native_timezone: this is working perfectly in my app. events are created sucessfully and display into my device calendar.

Mine is not working. even the imports

please add following depandancy in your project
device_calendar: ^4.3.0
flutter_native_timezone:

var currentLocation = getLocation(await FlutterNativeTimezone.getLocalTimezone()); setLocalLocation(currentLocation); print(currentLocation);
await Future.delayed(const Duration(seconds: 2));
var eventToCreate = Event(
  "1",
  //  eventId: "50",
  title: "calendarEventModel 12345",
  description: "eventDescription",
  start: TZDateTime.now(currentLocation),
  end: TZDateTime.now(currentLocation).add(Duration(hours: 10)),
);

final createEventResult =
    await deviceCalendarPlugin.createOrUpdateEvent(eventToCreate);
if (createEventResult!.isSuccess) {
  Utils.showTopMessage("Event added to Calendar", Colors.green);
  Get.back();
}

}
i used following packages device_calendar: ^4.3.0 flutter_native_timezone: this is working perfectly in my app. events are created sucessfully and display into my device calendar.

Mine is not working. even the imports

please add following depandancy in your project device_calendar: ^4.3.0 flutter_native_timezone:

Okay,  I'll try it. Thanks

Where is device deviceCalendarPlugin from? I cant see to find it

Also, I need the start and end date to be of a DateTime type. How can I achieve that?

I managed to get it working but after event has been created successfully, I don't see it in my calendar

I managed to get it working but after event has been created successfully, I don't see it in my calendar

same issue here on latest version

I too to get it working but after event has been created successfully, I don't see it in my calendar,
but if eventID remove then it will show, but we want to delete a event then how will delete without id?


Add apiextensions to scheme

https://github.com/stolostron/backlog/issues/22006
Signed-off-by: Philip Wu<EMAIL_ADDRESS>

/lgtm

/approve


Panning error

Hello.
I have some point on the map. I'm trying to change the location of the example https://www.react-simple-maps.io/zoom-pan

I chose the point "Сибирский ФО". Unfortunately the map flies off the screen. What I am doing wrong?

I had such settings:
projectionConfig={{
    scale: 280,
    rotation: [-10, 0, -40],
}}

I set
projectionConfig={{
    scale: 280,
    rotation: [-10, 0, 0],
}}

It works, but the map itself looks a bit wrong


Update publish.yml workflow

Relates to #381, #382
Summary
Update the "Publish / npm" job to explicitly set --access public following the example workflow seen in: https://docs.npmjs.com/generating-provenance-statements

Superseded by #384


Rewrite ChatMessages component using composition API

Move the logic responsible for sending prompts to bots and updating messages to the store.
Remove the direct reference to the ChatMessages component.

Sorry for breaking down the Matomo server in the previous commit. This commit fixes the problems. Now it should work as expected.

merged manually


[BEAM-14525] / #21626 Fix for Protobuf getter/setter method name discovery issue

https://issues.apache.org/jira/browse/BEAM-14525
Closes #21626.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
~[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.~

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

R: @aaltay

R: @reuvenlax - and in case he does not have time, R: @kileys

Thank you for reviewing @kileys !

LGTM. Could you resolve conflict and update the Jira to Github issue? Thanks

Run Java PreCommit

@kileys  I'm done, and haven't checked the code that throws the failure. -.-

Run Java PreCommit

Run Java PreCommit

Run Java PreCommit


[New game]: Toast Notification

🎮 Game Request
Game logic and basic description : The toast notification game is a simple web-based game where clicking a button generates toast notifications with random messages and types. These notifications appear on the screen and disappear after a short duration.
Point down the features
Game points :

Randomized Content: Generates notifications with random messages and types (info, success, error) for variety.
Auto-Removal: Notifications disappear after 3 seconds, ensuring a clean UI.
Dynamic Creation: Clicking the button creates a new notification, enhancing interactivity.
Reusable Functions: Separate functions handle random message/type generation, making the code modular and easy to extend.

Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @Vipullakum007 , You can't have another issue before completing the previous one 😀
you already have the following 1 open issues 👀 ! :
#3582

Hey @Vipullakum007 ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇

Hey @Vipullakum007 !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

/assign


[kwokctl] Add stop/start commands

What would you like to be added?
Add stop command for cluster.
Why is this needed?
Mostly, I just want to stop the running of cluster when I don't need it, NOT deleting.

/triage accepted
/good-first-issue

/retitle [kwokctl] Add stop/start commands

/assign

@wzshiming
I have a question, stop mean clean all containers or stop containers? The difference between the two is that if you are cleaning the container, the data created after startup will not exist.
Which is it?

@ikaven1024 @Fish-pro
I feel this requirements like a combination of two commands
kwokctl stop cluster
kwokctl snapshot save --path db.bak
kwokctl delete cluster

kwokctl start cluster
kwokctl create cluster
kwokctl snapshot restore --path db.bak

Cann't we just stop containers, keep the data volume? Just like minikube
~ minikube stop cluster -h
Stops a local Kubernetes cluster. This command stops the underlying VM or container, but keeps user
data intact. The cluster can be started again with the "start" command.


Card.onExpandChange called twice

When I set both 'actAsExpander' and 'showExpandableButton' of CardHeader, the onExpandChange function will be called twice when clicking the expand button.
Is it an issue to be fixed, or a feature as it is?

I don't think its an issue. I am closing this for now if you feel you need further help with this do let me know i will reopen this.

It looks to me like this issue still exists. See https://github.com/callemall/material-ui/blob/v0.15.4/src/Card/Card.js#L106-L114
In the event that showExpandableButton and and actAsExpander are true, the child (CardHeader) will receive handleExpanding as the prop touchTapEvent AND CardExpandable will receive handleExpanding as the prop onExpandChange. Both of these will be triggered by a click on the CardExpandable component. I think this issue could be solved by adding event.stopPropogation() to the handleExpanding function.

@tbescherer ,I agree with you.
My demo Code :
`

handleChange(){
console.log('change');
}
handleClick(e){
console.log('click');
e.stopPropagation();
}
`
I have two issue:
1:I can not find any props to modify showExpandableButton's style in

2：
the onExpandChange is always invoked before handleClick, which means I can not use stopPropagation. and just like @rogerwq, onExpandChangewill be triggered twice if
ExpandableButton clicked.

this seems to persist in ~0.16.4

^
It persists in 0.16.7. too.

I'm using Lodash debounce method as a temporary workaround. It prevents my onChange handler function from being called twice.
<Card onExpandChange={debounce(this.onExpandChange, 150)}>
  ...
</Card>

the issue still persists

Given that this issue seems to still exist, can someone explain why this was closed? Can someone re-open this?  I'd prefer not to create a new issue just to replicate the content.


fix(aws-applicationautoscaling): StepScalingPolicy missing ScalingTargetId and AdjustmentType

Fix StepScalingAction and StepScalingPolicy to add ScalingTargetId and default AdjustmentType to CloudFormation template.

Pull Request Checklist

[x] Testing

Unit test added (prefer not to modify an existing test, otherwise, it's probably a breaking change)
CLI change?: coordinate update of integration tests with team
cdk-init template change?: coordinated update of integration tests with team

[x] Docs

jsdocs: All public APIs documented
README: README and/or documentation topic updated
Design: For significant features, design document added to design folder

[x] Title and Description

Change type: title prefixed with fix, feat and module name in parens, which will appear in changelog
Title: use lower-case and doesn't end with a period
Breaking?: last paragraph: "BREAKING CHANGE: <describe what changed + link for details>"
Issues: Indicate issues fixed via: "Fixes #xxx" or "Closes #xxx"

[ ] Sensitive Modules (requires 2 PR approvers)

IAM Policy Document (in @aws-cdk/aws-iam)
EC2 Security Groups and ACLs (in @aws-cdk/aws-ec2)
Grant APIs (only if not based on official documentation with a reference)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Added comment regarding why I've added the properties. Cloudformation requires either the ResourceId, ScalableDimension, and ServiceNamespace properties, or the ScalingTargetId property, but not both.

Ah -- I meant to add the context in the commit message, not in the code itself. Given that we may want to add those other three fields in the future, I figured it made more sense to give context in the commit only.


Refactor: Alias

Current coverage is 74.73% (diff: 100%)

Merging #907 into master will decrease coverage by 0.06%

@@             master       #907   diff @@
==========================================
  Files           128        128
  Lines          6824       6803    -21
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           5104       5084    -20
+ Misses         1720       1719     -1
  Partials          0          0

Powered by Codecov. Last update 576231f...130befb


Create Posts “gwang-ho_humans-of-rangitoto”

Automatically generated by Netlify CMS

👷 Deploy Preview for rangitoto-reporter processing.

Name
Link

🔨 Latest commit
17d21c0afcd25389d9608098c52aa2bc8d519c2f

🔍 Latest deploy log
https://app.netlify.com/sites/rangitoto-reporter/deploys/627a26708c7bd40008211433


Add a workaround a Keycloak bug causing the test failures

https://github.com/snowdrop/secured-example/pull/68


StackOverflow when running C# samples

Hi,
I'm getting the following error when running the Parser's sample C# code from the site [1]:
An unhandled exception of type 'System.StackOverflowException' occurred in IKVM.OpenJDK.Core.dll
on line:
var gs = gsf.newGrammaticalStructure(tree2);
It's all good up to that point.
CoreNLP sample in C# also fails with StackOverflow exception on the following line if annotators include "parse":
pipeline.annotate(annotation);
// With this, it works: props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner");
Tests in F# are running fine. I'm using the same CoreNLP library downloaded under "\paket-files\nlp.stanford.edu\stanford-corenlp-full-2015-01-30\models" through build.cmd.
I couldn't be able to identify any differences between C# sample and F# tests so I'm quite stuck figuring what's going on. Any help would be appreciated.
Thanks.
[1] http://sergey-tihon.github.io/Stanford.NLP.NET/StanfordParser.html

@altinoren Do you run code equal to samples from site? (without any changes)
@volhav Could you assist here?

C# sample for Stanford Parser works on my machine
https://github.com/sergey-tihon/Stanford.NLP.NET/commit/549a59c86beca7ac0d5dfb29f1d780bd0fc5ffc2

@altinoren I've run both samples as well and they work fine on my machine. Could you please check that you have latest package version from Nuget? If so, could you show your console output? Not only error I mean. Probably it'll help us understand what is going wrong

MY report above is correct but apparently missing some details.
I was testing it in an ASP.NET MVC application. After you request console output, I tested the code in a console application and all works fine (I should have done this in the first place, sorry. I wasn't expecting such a difference based on the platform.). On MVC, not only it raises the error but also considerably slow, there must be some other factors based on the environment.
I'll test some more with a fully deployed IIS scenario and let you know.

It is strange because I use it under IIS for years.
It worked under .NET 3.5 and .NET 4.5 for me.

IKVM.Net breaks when hosted on IIS Express (on 4.5 Framework, run directly from Visual Studio), and all works well when deployed to IIS. I won't devote more time investigating further since I'll work with Stanford.NLP.NET on IIS for my actual project.
If I have time to test it on a machine other than mine, and if the problem continues, I'll report it to IKVM.Net project.
Thank you all for your time.

Thanks for solving the problem. Indeed, IKVM can't work with IIS express and works perfectly on IIS.

Do you understand why it does not work under IIS Express?

No idea :)


Due to the age of the list, many entries have become obsolete/deprecated. Should these get a new separate category for obsolete stuff or should we remove them completely?
I've had an issue brought up about some entries potentially breaking Riot TOS (like #49). Should we start policing these?
deciding order in which entries are sorted? alphabetical, time of adding, something else?
potentially refactoring the way we categorize entries?
switching the branch from master to main (technically an official requirement if we ever wanna be added to awesome-list repo)

This is something I'd love to get a feedback about from everyone, contributor or not, readers, feel free to drop your input

There's a few things I'd like to get done and to decide on some matters.

Due to the age of the list, many entries have become obsolete/deprecated. Should these get a new separate category for obsolete stuff or should we remove them completely?
I've had an issue brought up about some entries potentially breaking Riot TOS (like Added LeagueAutoAccept #49). Should we start policing these?
deciding order in which entries are sorted? alphabetical, time of adding, something else?
potentially refactoring the way we categorize entries?
switching the branch from master to main (technically an official requirement if we ever wanna be added to awesome-list repo)

This is something I'd love to get a feedback about from everyone, contributor or not, readers, feel free to drop your input

My personal thoughts on these points:

Remove them completely, or perhaps have a deprecated/legacy.md for them.
Perhaps to an extent, but in that case where would the line be drawn for fine/no-no. I'm not sure, it also kind of depends on the impact the tool would have. I don't think tools that might be likely to get the player banned should be a listed (don't think any of the current listed fall under that). So...  🤷♂?
Alphabetical or maybe by stars.
Distinguishing the "Applications" and "External Apps" more wouldn't be a bad idea so that there are no more websites under "Applications". And for ones that offer both, like Blitz for example, you could list them under both as long as it provides value for both (e.g. don't put something under "External Apps"/"Websites" if the website is like Nemo, where the website purpose is just to provide the download & contact info)
No real reason not to do this

Something else: Maybe it'd be a nice idea to let listings that have a website URL but are also open-source have a little :octocat: GH icon next to it linking to the repository. (Like https://mimic.lol/ and https://github.com/molenzwiebel/mimic)

deciding order in which entries are sorted? alphabetical, time of adding, something else?

When I first contributed, I was a bit confused about how I should add my entry based on the ordering. Maybe it's best to have them sorted alphabetically. x3

I've had an issue brought up about some entries potentially breaking Riot TOS (like https://github.com/CommunityDragon/awesome-league/pull/49). Should we start policing these?

I'm more on the side of policing because just having apps like these here implies that it's being endorsed. I just don't know how to go on about it though.  :>

I've had an issue brought up about some entries potentially breaking Riot TOS (like https://github.com/CommunityDragon/awesome-league/pull/49). Should we start policing these?

I agree with @BlossomiShymae on this one

Distinguishing the "Applications" and "External Apps" more wouldn't be a bad idea so that there are no more websites under "Applications". And for ones that offer both, like Blitz for example, you could list them under both as long as it provides value for both (e.g. don't put something under "External Apps"/"Websites" if the website is like Nemo, where the website purpose is just to provide the download & contact info)

I really like this, as some apps are in the wrong category and sorting them (and make it clearer for further contributors) would help a lot.

I'm more on the side of policing because just having apps like these here implies that it's being endorsed. I just don't know how to go on about it though. :>

I also agree with this, especially as TOS-breaking apps could get the users banned, which should be nothing that is getting endorsed here.

Alright a draft from me of all the changes to do:

[ ]  Aforementioned LeagueAutoAccept will be removed, will snipe any other similarly functioning apps per the "functioning on user behalf" part. I also looked through issues, seems there's been a one KR server suspension for using this. (albeit they are way stricter about LCU access)

[ ]  Categories: Client Applications, External Applications, Content Creation, Developer Tools, Developer Documentation (gonna merge devops tools into that probably, both categories are miniscule anyways)
[ ]  master to main, it should just require workflow adjustments and there's really no reason to not do it
[ ]  moving deprecated entries to a separate markdown file for historical(?) reasons
[ ] updating contribution guidelines to mention all the riot guidelines so we avoid tosbreaking tools
[ ] alphabetical list orders, might try if i can make a workflow for validating that
[ ] separate source link for OSS

lemme know if i missed something or would like me to adjust something


Implement use argument for LLM for decorators

Describe your changes and why you are making these changes
This PR implements the use argument that allows the user to specify the intent to use our LLM library. Under the hood, we include the use_llm flag into the resource struct and use this as a flag to select Docker images that have LLM Python library dependencies and Cuda drivers pre-installed.
Related issue number (if any)
Loom demo (if any)
Checklist before requesting a review

[ ] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
[ ] I have performed a self-review of my code.
[ ] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
[ ] If this is a new feature, I have added unit tests and integration tests.
[ ] I have run the integration tests locally and they are passing.
[ ] I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
[ ] All features on the UI continue to work correctly.
[ ] Added one of the following CI labels:

run_integration_test: Runs integration tests
skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

@hsubbaraj-spiral updated the implementation to add the llm_op API shown in this demo. If you can take a look at llm_wrapper.py that'd be great.


[Quota] Update pre-populated Quota tariffs' type

Description
The PR #7152 changed the unit of the Quota types VM_DISK_IO_READ, VM_DISK_IO_WRITE, VM_DISK_BYTES_READ and VM_DISK_BYTES_WRITE. However, the pre-populated tariffs were not considered, and their unit type is still with the old value. The unit was only displayed incorrectly, as the calculation utilizes the correct unit. Their unit types were updated to the correct ones.
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale or Bug Severity
Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[x] Minor
[ ] Trivial

How Has This Been Tested?
Update script
2024-02-05 14:42:03,890 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) DB version = <IP_ADDRESS> Code Version = <IP_ADDRESS>
2024-02-05 14:42:03,894 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Database upgrade must be performed from <IP_ADDRESS> to <IP_ADDRESS>
2024-02-05 14:42:03,949 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Running upgrade Upgrade41900to41910 to upgrade from <IP_ADDRESS>-<IP_ADDRESS> to <IP_ADDRESS>
2024-02-05 14:42:03,957 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Licensed to the Apache Software Foundation (ASF) under one
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- or more contributor license agreements.  See the NOTICE file
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- distributed with this work for additional information
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- regarding copyright ownership.  The ASF licenses this file
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- to you under the Apache License, Version 2.0 (the
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "License"); you may not use this file except in compliance
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- with the License.  You may obtain a copy of the License at
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --   http://www.apache.org/licenses/LICENSE-2.0
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Unless required by applicable law or agreed to in writing,
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- software distributed under the License is distributed on an
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- KIND, either express or implied.  See the License for the
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- specific language governing permissions and limitations
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- under the License.
2024-02-05 14:42:03,964 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:03,964 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Schema upgrade from <IP_ADDRESS> to <IP_ADDRESS>
2024-02-05 14:42:03,965 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:03,965 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Updates the populated Quota tariff's types VM_DISK_BYTES_READ, VM_DISK_BYTES_WRITE, VM_DISK_IO_READ and VM_DISK_IO_WRITE to the correct unit.
2024-02-05 14:42:03,966 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) UPDATE cloud_usage.quota_tariff SET usage_unit = 'Bytes', updated_on = NOW() WHERE effective_on = '2010-05-04 00:00:00' AND name IN ('VM_DISK_BYTES_READ', 'VM_DISK_BYTES_WRITE')
2024-02-05 14:42:03,976 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) UPDATE cloud_usage.quota_tariff SET usage_unit = 'IOPS', updated_on = NOW() WHERE effective_on = '2010-05-04 00:00:00' AND name IN ('VM_DISK_IO_READ', 'VM_DISK_IO_WRITE')
2024-02-05 14:42:04,091 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup upgrade Upgrade41900to41910 to upgrade from <IP_ADDRESS>-<IP_ADDRESS> to <IP_ADDRESS>
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Licensed to the Apache Software Foundation (ASF) under one
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- or more contributor license agreements.  See the NOTICE file
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- distributed with this work for additional information
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- regarding copyright ownership.  The ASF licenses this file
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- to you under the Apache License, Version 2.0 (the
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "License"); you may not use this file except in compliance
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- with the License.  You may obtain a copy of the License at
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --   http://www.apache.org/licenses/LICENSE-2.0
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Unless required by applicable law or agreed to in writing,
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- software distributed under the License is distributed on an
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- KIND, either express or implied.  See the License for the
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- specific language governing permissions and limitations
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- under the License.
2024-02-05 14:42:04,107 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:04,107 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Schema upgrade cleanup from <IP_ADDRESS> to <IP_ADDRESS>
2024-02-05 14:42:04,108 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:04,115 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup script Upgrade41900to41910 is executed successfully
2024-02-05 14:42:04,225 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Upgrade completed for version <IP_ADDRESS>

After the update script, the usage units were updated accordingly.
|id|usage_type|usage_name|usage_unit|usage_discriminator|currency_value|effective_on|updated_on|updated_by|uuid|name|description|activation_rule|removed|end_date|
|--|----------|----------|----------|-------------------|--------------|------------|----------|----------|----|----|-----------|---------------|-------|--------|
|18|21|VM_DISK_IO_READ|IOPS|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2dc7-771e-11ee-8e59-5254003754dc|VM_DISK_IO_READ|||||
|19|22|VM_DISK_IO_WRITE|IOPS|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2ead-771e-11ee-8e59-5254003754dc|VM_DISK_IO_WRITE|||||
|20|23|VM_DISK_BYTES_READ|Bytes|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2f84-771e-11ee-8e59-5254003754dc|VM_DISK_BYTES_READ|||||
|21|24|VM_DISK_BYTES_WRITE|Bytes|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f304f-771e-11ee-8e59-5254003754dc|VM_DISK_BYTES_WRITE|||||

How did you try to break this feature and the system with this change?

@JoaoJandre, do you think this can go to the <IP_ADDRESS> release, as it is a small fix?

@blueorangutan package

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 8489

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

@JoaoJandre
You need to add this to the upgrade schema of both 4.18.1-4.18.2 and 4.19.0-4.19.1
Considering this is a minor fix and this is the only database change between 4.18.1 and 4.18.2, can we retarget to 4.19.1 and add it to the upgrade schema of 4.19.0-4.19.1 only ?

@blueorangutan package

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 8533

@blueorangutan test

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian test result (tid-9257)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56799 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8591-t9257-kvm-centos7.zip
Smoke tests completed. 120 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_list_volumes_metrics_history
Error
3607.52
test_metrics_api.py

test_02_trigger_shutdown
Failure
336.64
test_safe_shutdown.py

some CI failures, reopening.

still doesn't pass :( though the error is in a library download. Applying insanity

@blueorangutan package

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 8844

@blueorangutan test

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian test result (tid-9380)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44248 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8591-t9380-kvm-centos7.zip
Smoke tests completed. 129 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File


[docs] Minor fixes for subsection links

@Bradamant3 PTAL thanks!
There might be other broken links that need to be corrected---the results of the link checker test were a bit noisy (since I was testing the raw output of the site gen tool). Figured that I might as well fix these ones though!

/lgtm and thanks!
(also weird, bc I could have sworn I had these same changes in a commit somewhere. Evidently did not make it in ;) )

Also @ncdc @skriss I'll get the site rebuilt with these changes sometime today. It's a little fiddly to fix up the 0.7.0 branch, but it's also a good opportunity to make sure my assumptions about what to do make sense going forward.

@Bradamant3 I'm not sure how the tooling works, but we could maybe have it check to see if there's a release-MAJOR-MINOR branch to match a tag's major & minor versions, and if there is, use that instead of the tag?
For example, if you need to adjust what's published for 0.7.x docs, and it doesn't require any ark code changes in git, then we could create a release-0.7 branch, you could make your copy edits there, and the tooling could see there's a v0.7.0 tag, see there's a release-0.7 branch, and build off the branch. Do you think that would be a useful feature?

Closing because these changes made it in in other PRs. Tracking suggestion from @ncdc in issue in docs tooling repo.


Design the Authentication/Authorization mechanisms of Corpora

AuthN and AuthZ will be a shared service across the data portal and cellxgene. Will need to coordinate with the cellxgene team.
This ticket should culminate in a design doc approved by engineers on both efforts.

For M2 planning, there are pending design discussions in the 2020 plan related to how sharing capability url(s) may intersect with authentication:

Comment Thread

Comment Thread

Capturing notes from design discussion:
The 2020 plan will be updated to reflect the following changes. Registration is only required for two cases:

To submit and manage datasets
To create and persist user annotations or other user state

Registration is NOT required for consumers downloading datasets or viewing projects or data.
For Scientists who host pre-publication data on Corpora want their data to be private until their dataset has been peer reviewed so they have a chance to fix errors before the data are widely viewed.:
Submitters may request one capability url (bearer token) per:

submitted private project
pending submission to revise a private submitted project

Capability urls are manually forwarded to reviewers or collaborators by the submitter to authorize access to a submitted private project or its pending revision. Note, this is coarse granularity because the capability url is per project/revision and not per reviewer/collaborator. No registration/authentication is required when a user is accessing a project with a capability url.
Projects owners may revoke a capability url at will. When a user navigates to a revoked url, there should be a “friendly” explanation offered rather than 404.
The capability url for a submitted private project must be constant unless the owner revokes the url. As revisions are submitted, the private project content changes but its capability url does not.
Once pending revisions are submitted, the capability url for the revision is automatically revoked. “It disappears”.
When a private project is published, the capability url for the private version must not be automagically revoked by the system. The owner is responsible for manually revoking “at the right time” which may be never. (There may be ongoing reviews of the private submission which would be interrupted)

Progress so far.  I've been coming up to speed with Auth0 by implementing a prototype for single page apps and one for regular web app.   There are a few more questions about how specifically to do the authentication challenge in a way that does not overly disrupt a user's progress in cellxgene.  I plan to do another experiment to resolve this.   By next Wednesday I plan to have a design document ready.

The first pass at the design document is complete, and In review
https://docs.google.com/document/d/19cC_tH7DOHIYqojdo-6T9eTzPmoMkwrAj83Ozvqth8s/edit?usp=sharing


36quota defeating

removed all old api methods
added four new api methods:

interestByRegion
interestOverTime
relatedQueries
relatedTopics

removed all dependencies
adding codeclimate
fixing coveralls
removed old tests and added new tests for new methods
adding eslint and following airbnb style guide
updated README accordingly

Changes Unknown when pulling 3076985d06cd48a17b9cf47e6bb5df863458fb0c on 36quotaDefeating into ** on master**.

Changes Unknown when pulling 01e92b14c73406ef0d25d852c6b8886fc04893f3 on 36quotaDefeating into ** on master**.


ECDH/TLS Proxy Demo bugfixes and minor refactor

This pull request incorporates a number of modifications to the ECDH/TLS proxy demo:

In an attempt to resolve some issues being encoundered in the pelz test code which leverages this code, this is an extension to pull request #186. It proposes changing the name used in server certificate verification to:

<network host name or IP address for server>.<a unique name that can indicate the nodes functionality>.

For the ECDH/TLS proxy node, this required adding a command line option to specify the desired 'functional name'.
A little bit of code was added to concatenate the host/IP and name parameters,
the Makefile had to be updated to specify the new "--remote-name" parameter, and
the test key configuration scripts had to be updated to specify this 'verification name' as a subject alternate name (SAN) in the form of a DNS name entry.
This changes some of the member names in the TLS context struct, to hopefully be a little clearer (i.e., host => remote_server, remote_server_func (new), and port => conn_port),

Changes the test key generation configuration to specify SHA-512 instead of SHA-256

Adds SSL_VERIFY_FAIL_IF_NO_PEER_CERT to the certificate verification configuration. This should force a 'handshake failure' if the TLS client does not supply the TLS server with the requested certificate.

Although the demo appeared to be completely successful, there were segfaults in the log. This was tracked down to an improper ordering of the two parameters for a call to demo_tls_init() called during end of program "cleanup", so this was fixed.

Was not receiving the "normal termination" message for the main (parent) process of the proxy. After investigating, this was expected behavior, but I expanded the debug logging slightly to provide better visibility into this. I also moved the exit point for child processes handling a session to the end of the proxy_handle_session() function. I left the 'return' at the end of main(), but added logging to indicate unexpected termination and change the return value to EXIT_FAILURE.

For the demo simplified KMIP server, added a call the the 'cleanup' function before exiting due to the help (-h) command-line option. If additional options were specified, there could potentially be some memory/resources allocated that need to be freed and/or cleared.

The demo simplified KMIP server was not calling the demo_tls_server_accept() before BIO_do_handshake(), so this was added.

In testing, I discovered that client certificate verification errors occur after the TLS handshake is complete (e.g., during the BIO_do_read() call. A few changes were made to correct comments that indicated it occurred earlier and expand logging to better report when a client certificate verification failure occurs.

Removed Server Name Indication (SNI) configuration for the TLS client. The simplified demo server does not implement this functionality.

demo_tls_init() and demo_ecdh_init() were being called twice during program cleanup (once by the ECDH/TLS context cleanup function and once by the proxy cleanup function). I initially removed them from the ECDH/TLS context cleanup, but then added them back in because there could potentially be a chance that these functions may not always be used together in the same way they are for the demo.

For the demo simplified KMIP server, the context was not getting initialized, so this was added

A few whitespace edits were made to address things like long lines in the source code

Note: after submitting and re-reading, I added a commit to change the renamed variable names in the TLS context.

Commit a05483a incorporates a 'git rebase' of the 'main' branch to add in the recently merged (since this pull request was submitted) changes from PR #190 and PR #191.
This was probably unnecessary as this will hopefully eventually get merged into main anyway, but it cleans up some build warnings in the meantime.


Vineyard - Airflow Celery Executor: Can Not Transfer Dataframe Between Tasks

Describe your problem
A clear and concise description of what your problem is. It might be a bug,
a feature request, or just a problem that need support from the vineyard team.

If is is a bug report, to help us reproducing this bug, please provide information below:

Your Operation System version (uname -a):

Master:   Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-77-generic x86_64)
Worker 1 & 2: Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-89-generic x86_64)

The version of vineyard you use (vineyard.__version__):  0.3.16
Versions of crucial packages, such as gcc, numpy, pandas, etc.:
pandas=1.3.1  numpy=1.21.1   airflow=2.1.0  gcc=(Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Full stack of the error (if there are a crash):

*** Log file does not exist: /data/airflow/logs/taskflow_etl_pandas/extract/2022-01-25T11:54:00.395325+00:00/1.log
*** Fetching from: http://DTEKAIRFLOWWP1.fw.dteknoloji.com.tr:8793/log/taskflow_etl_pandas/extract/2022-01-25T11:54:00.395325+00:00/1.log
[2022-01-25 14:54:01,671] {taskinstance.py:876} INFO - Dependencies all met for <TaskInstance: taskflow_etl_pandas.extract 2022-01-25T11:54:00.395325+00:00 [queued]>
[2022-01-25 14:54:01,690] {taskinstance.py:876} INFO - Dependencies all met for <TaskInstance: taskflow_etl_pandas.extract 2022-01-25T11:54:00.395325+00:00 [queued]>
[2022-01-25 14:54:01,690] {taskinstance.py:1067} INFO -
[2022-01-25 14:54:01,690] {taskinstance.py:1068} INFO - Starting attempt 1 of 1
[2022-01-25 14:54:01,690] {taskinstance.py:1069} INFO -
[2022-01-25 14:54:01,699] {taskinstance.py:1087} INFO - Executing <Task(_PythonDecoratedOperator): extract> on 2022-01-25T11:54:00.395325+00:00
[2022-01-25 14:54:01,704] {standard_task_runner.py:52} INFO - Started process 2502514 to run task
[2022-01-25 14:54:01,709] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'taskflow_etl_pandas', 'extract', '2022-01-25T11:54:00.395325+00:00', '--job-id', '225', '--pool', 'default_pool', '--raw', '--subdir', 'DAGS_FOLDER/test_df_transfer_between_tasks.py', '--cfg-path', '/tmp/tmpk31jbrbm', '--error-file', '/tmp/tmpi098c023']
[2022-01-25 14:54:01,710] {standard_task_runner.py:77} INFO - Job 225: Subtask extract
[2022-01-25 14:54:01,783] {logging_mixin.py:104} INFO - Running <TaskInstance: taskflow_etl_pandas.extract 2022-01-25T11:54:00.395325+00:00 [running]> on host DTEKAIRFLOWWP1.fw.dteknoloji.com.tr
[2022-01-25 14:54:01,868] {taskinstance.py:1280} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=taskflow_etl_pandas
AIRFLOW_CTX_TASK_ID=extract
AIRFLOW_CTX_EXECUTION_DATE=2022-01-25T11:54:00.395325+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-01-25T11:54:00.395325+00:00
[2022-01-25 14:54:01,875] {python.py:151} INFO - Done. Returned value was: ['a', 'b']
[2022-01-25 14:54:01,875] {xcom.py:228} ERROR - Could not serialize the XCom value into JSON. If you are using pickle instead of JSON for XCom, then you need to enable pickle support for XCom in your airflow config.
[2022-01-25 14:54:01,876] {taskinstance.py:1481} ERROR - Task failed with exception
Traceback (most recent call last):
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1137, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1311, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1344, in _execute_task
self.xcom_push(key=XCOM_RETURN_KEY, value=result)
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1919, in xcom_push
XCom.set(
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/utils/session.py", line 67, in wrapper
return func(*args, **kwargs)
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/xcom.py", line 79, in set
value = XCom.serialize_value(value)
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/xcom.py", line 226, in serialize_value
return json.dumps(value).encode('UTF-8')
File "/data/anaconda3/envs/airflow/lib/python3.8/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/data/anaconda3/envs/airflow/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/data/anaconda3/envs/airflow/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/data/anaconda3/envs/airflow/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type DataFrame is not JSON serializable
[2022-01-25 14:54:01,877] {taskinstance.py:1524} INFO - Marking task as FAILED. dag_id=taskflow_etl_pandas, task_id=extract, execution_date=20220125T115400, start_date=20220125T115401, end_date=20220125T115401
[2022-01-25 14:54:01,918] {local_task_job.py:151} INFO - Task exited with return code 1

Minimized code to reproduce the error:

import numpy as np
import pandas as pd

from airflow.decorators import dag, task
from airflow.utils.dates import days_ago

default_args = {
    'owner': 'airflow',
}

@dag(default_args=default_args, schedule_interval=None, start_date=days_ago(2), tags=['example'])
def taskflow_etl_pandas():
    @task()
    def extract():
        df = pd.DataFrame({
            'a': np.random.rand(100000),
            'b': np.random.rand(100000),
        })
        return df

    @task(multiple_outputs=True)
    def transform(dataframe):
        return {"total_order_value": dataframe["a"].sum()}

    @task()
    def load(total_order_value: float):
        print(f"Total order value is: {total_order_value:.2f}")

    order_data = extract()
    order_summary = transform(order_data)
    load(order_summary["total_order_value"])

taskflow_etl_pandas_dag = taskflow_etl_pandas()

Hello, I am trying to integrate vineyard backend on airflow cluster, however it seems like airflow still uses default xcom backend as seen on error;
File "/data/anaconda3/envs/airflow/lib/python3.8/site-packages/airflow/models/xcom.py", line 226, in serialize_value
I will explain steps I did to configure backend of my cluster.

Installed vineyard on all three nodes on airflow environment:  pip install airflow-provider-vineyard
Added last 2 lines to my airflow.env file which is in /data/aiflow/ folder. There are  4 env variables in  airflow.env file now. Did same on all 3 nodes.

AIRFLOW_CONFIG=/data/airflow/airflow.cfg
AIRFLOW_HOME=/data/airflow
AIRFLOW__CORE__XCOM_BACKEND=vineyard.contrib.airflow.xcom.VineyardXCom
AIRFLOW__VINEYARD__IPC_SOCKET=/tmp/vineyard.sock

Launched vineyard server on all nodes: nohup python -m vineyard --socket=/tmp/vineyard.sock
Checked that 2379, 2380 and 6900 vineyard ports are being listened and every single node has telnet connection to these ports on other 2 nodes.

Restarted webserver and scheduler services on master node.
systemctl restart airflow-webserver
systemctl restart airflow-scheduler

After I got fail on airflow task with these default settings, I tried some other configurations:
I found out that, at step 2,  vineyard.contrib.airflow.xcom.VineyardXCom path seems missing backend.py between xcom folder and VineYardXCom class so I changed this line to AIRFLOW__CORE__XCOM_BACKEND=vineyard.contrib.airflow.xcom.backend.VineyardXCom

Later I added deployement setting to disitributed on launching command:
nohup python -m vineyard --socket=/tmp/vineyard.sock --deployment=distributed
Finally, I have changed xcom_backend setting pointing to VineyardXCom class on airflow.cfg file.
'#Path to custom XCom class that will be used to store and resolve operators results
"#Example: xcom_backend = path.to.CustomXCom
"#xcom_backend = airflow.models.xcom.BaseXCom'
xcom_backend = vineyard.contrib.airflow.xcom.backend.VineyardXCom
After each change, I re-launched vineyard server on all nodes and restarted airflow webserver and scheduler, however None of trials succeeded on airflow. still getting same error I shared at top.  Looking forward for your help. Thanks in advance.

Seems that the vineyard backend doesn't take into affect.
I'm sure if it is a regression bug that related to airflow's version. I will try to reproduce with airflow 2.1.0.

Thanks for quick response. If you dont see any incorrect or missing step on my configuration steps then it is highly likely related to airflow version. Because I also tried to set custom s3 backend and having exactly same error as I am trying to transfer dataframe between tasks. S3 Backend does not take into affect aswell.
Which airflow version do you suggest for me, latest one? Maybe I can upgrade airflow.

I wanted to share quick update. After I had upgraded airflow from 2.1.0 to 2.2.3, I could have activated vineyard backend. However, socket connection error has occured now. I checked vineyard services, all of them are up and running. Any advice ?

Log file does not exist: /data/airflow/logs/tutorial_taskflow_api_etl/extract/2022-01-31T19:55:58.987530+00:00/1.log
Fetching from: http://DTEKAIRFLOWWP1.fw.dteknoloji.com.tr:8793/log/tutorial_taskflow_api_etl/extract/2022-01-31T19:55:58.987530+00:00/1.log

[2022-01-31, 22:56:00 UTC] {taskinstance.py:1032} INFO - Dependencies all met for <TaskInstance: tutorial_taskflow_api_etl.extract manual__2022-01-31T19:55:58.987530+00:00 [queued]>
[2022-01-31, 22:56:00 UTC] {taskinstance.py:1032} INFO - Dependencies all met for <TaskInstance: tutorial_taskflow_api_etl.extract manual__2022-01-31T19:55:58.987530+00:00 [queued]>
[2022-01-31, 22:56:00 UTC] {taskinstance.py:1238} INFO -

[2022-01-31, 22:56:00 UTC] {taskinstance.py:1239} INFO - Starting attempt 1 of 1
[2022-01-31, 22:56:00 UTC] {taskinstance.py:1240} INFO -

[2022-01-31, 22:56:00 UTC] {taskinstance.py:1259} INFO - Executing <Task(_PythonDecoratedOperator): extract> on 2022-01-31 19:55:58.987530+00:00
[2022-01-31, 22:56:00 UTC] {standard_task_runner.py:52} INFO - Started process 3266293 to run task
[2022-01-31, 22:56:00 UTC] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'tutorial_taskflow_api_etl', 'extract', 'manual__2022-01-31T19:55:58.987530+00:00', '--job-id', '473', '--raw', '--subdir', '/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/example_dags/tutorial_taskflow_api_etl.py', '--cfg-path', '/tmp/tmp47kohjns', '--error-file', '/tmp/tmp37ckf_hd']
[2022-01-31, 22:56:00 UTC] {standard_task_runner.py:77} INFO - Job 473: Subtask extract
[2022-01-31, 22:56:00 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: tutorial_taskflow_api_etl.extract manual__2022-01-31T19:55:58.987530+00:00 [running]> on host DTEKAIRFLOWWP1.fw.dteknoloji.com.tr
[2022-01-31, 22:56:00 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/vineyard/contrib/airflow/xcom/backend.py:147: SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
  for result in query.with_entities(VineyardXCom.value):

[2022-01-31, 22:56:00 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/vineyard/contrib/airflow/xcom/backend.py:156: SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
  query.delete()

[2022-01-31, 22:56:00 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/renderedtifields.py:126: SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
  session.merge(self)

[2022-01-31, 22:56:00 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/renderedtifields.py:197: SAWarning: Coercing Subquery object into a select() for use in IN(); please pass a select() construct explicitly
  tuple_(cls.dag_id, cls.task_id, cls.execution_date).notin_(subq2),

[2022-01-31, 22:56:00 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/renderedtifields.py:194: SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
  session.query(cls).filter(

[2022-01-31, 22:56:00 UTC] {taskinstance.py:1424} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=tutorial_taskflow_api_etl
AIRFLOW_CTX_TASK_ID=extract
AIRFLOW_CTX_EXECUTION_DATE=2022-01-31T19:55:58.987530+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-01-31T19:55:58.987530+00:00
[2022-01-31, 22:56:00 UTC] {python.py:175} INFO - Done. Returned value was: {'1001': 301.27, '1002': 433.21, '1003': 502.22}
[2022-01-31, 22:56:10 UTC] {taskinstance.py:1700} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
    self._execute_task_with_callbacks(context)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
    result = self._execute_task(context, self.task)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
    self.xcom_push(key=XCOM_RETURN_KEY, value=result)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2135, in xcom_push
    XCom.set(
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/utils/session.py", line 67, in wrapper
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/vineyard/contrib/airflow/xcom/backend.py", line 87, in set
    value = VineyardXCom.serialize_value(value)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/vineyard/contrib/airflow/xcom/backend.py", line 160, in serialize_value
    client = vineyard.connect(VineyardXCom.options()['ipc_socket'])
vineyard._C.ConnectionFailedException: Connection failed: Failed to connect to vineyardd:
[2022-01-31, 22:56:10 UTC] {taskinstance.py:1267} INFO - Marking task as FAILED. dag_id=tutorial_taskflow_api_etl, task_id=extract, execution_date=20220131T195558, start_date=20220131T195600, end_date=20220131T195610
[2022-01-31, 22:56:10 UTC] {standard_task_runner.py:89} ERROR - Failed to execute job 473 for task extract
Traceback (most recent call last):
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
    args.func(args, dag=self.dag)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
    return f(*args, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 298, in task_run
    _run_task_by_selected_method(args, dag, ti)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
    _run_raw_task(args, ti)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
    ti._run_raw_task(
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
    self._execute_task_with_callbacks(context)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
    result = self._execute_task(context, self.task)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
    self.xcom_push(key=XCOM_RETURN_KEY, value=result)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2135, in xcom_push
    XCom.set(
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/utils/session.py", line 67, in wrapper
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/vineyard/contrib/airflow/xcom/backend.py", line 87, in set
    value = VineyardXCom.serialize_value(value)
  File "/data/anaconda3/envs/airflow2/lib/python3.9/site-packages/vineyard/contrib/airflow/xcom/backend.py", line 160, in serialize_value
    client = vineyard.connect(VineyardXCom.options()['ipc_socket'])
vineyard._C.ConnectionFailedException: Connection failed: Failed to connect to vineyardd:
[2022-01-31, 22:56:10 UTC] {local_task_job.py:154} INFO - Task exited with return code 1
[2022-01-31, 22:56:10 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2022-01-31, 22:56:10 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/dag.py:1076: SADeprecationWarning: Query.value() is deprecated and will be removed in a future release.  Please use Query.with_entities() in combination with Query.scalar() (deprecated since: 1.4)
  return qry.value(DagModel.is_paused)

[2022-01-31, 22:56:10 UTC] {warnings.py:109} WARNING - /data/anaconda3/envs/airflow2/lib/python3.9/site-packages/airflow/models/dag.py:1076: SADeprecationWarning: Query.values() is deprecated and will be removed in a future release.  Please use Query.with_entities() (deprecated since: 1.4)
  return qry.value(DagModel.is_paused)

You need to start vineyard daemon (on airflow workers) before executing airflow tasks.
See also https://github.com/v6d-io/v6d/blob/main/python/vineyard/contrib/airflow/README.md

Configure Vineyard locally

The vineyard server can be easier launched locally with the following command:
python3 -m vineyard --socket=/tmp/vineyard.sock

Close as addressed.


Remove outdated dependencies

Removes this annoying warning:
\Warning: the following project(s) have [[constraint]] stanzas in Gopkg.toml:

  ✗  github.com/gorilla/websocket
  ✗  gopkg.in/tylerb/graceful.v1

However, these projects are not direct dependencies of the current project:
they are not imported in any .go files, nor are they in the 'required' list in
Gopkg.toml. Dep only applies [[constraint]] rules to direct dependencies, so
these rules will have no effect.

Either import/require packages from these projects so that they become direct
dependencies, or convert each [[constraint]] to an [[override]] to enforce rules
on these projects, if they happen to be transitive dependencies,

Pull Request Test Coverage Report for Build 557

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 97.5%

Totals

Change from base Build 555:
0.0%

Covered Lines:
273

Relevant Lines:
280

💛  - Coveralls


[BUG] Link output from bootstrap does not work if user has access to multiple tenants

Describe the bug
When the script New-SovereignLandingZone.ps1 is complete, a link to the portal is written to the output. If the deploying user has access to more than one tenant the code (Get-Tenant).Id returns multiple tenant ids which would be joined with space.
To Reproduce
Steps to reproduce the behavior:

Use an account with access to more than one tenant.
Run ./New-SovereignLandingZone.ps1 -parDeployment bootstrap
The link outputted will not be clickable since it contains spaces that are not url encoded.

Expected behavior
The link should be clickable in the terminal

@SimonWahlin this issue has been resolved. Please validate against this latest code from main branch.


Added: __version__ attribute

add version attribute
add pyproyect.toml instead of setup.py, resolve #228
use pdm for managing, locking development dependencies instead of pip-compile
use pdm for package building instead of build
use pdm.scripts instead of makefile, resolve #197
use tox for testing with multiple python version, resolve #229
enable pylint missing-docstring only in the src
refactor the tools folder

Most of the modifications:

fix formatting with the newest version of prettier
keep docstring only in the src
rename data_frame variables to df

Important modification:

src/ipyvizzu/init.py
src/ipyvizzu/version.py
pyproject.toml (instead of setup.py)
pyproject.toml scripts (instead of Makefile)
CONTRIBUTING.md


Using rasa-core endpoints to connect rasa-nlu server gets an unurldecoded string.

Rasa NLU version:0.14.1
Operating system (windows, osx, ...):osx
Content of model configuration file:

Issue:rasa-core endpoints to connect rasa-nlu server got urlencodeed str

rasa-nlu's log is like this : it indicate that the rasa-core uses a GET request
2019-02-22 19:43:36+0800 [-] "<IP_ADDRESS>" - - [22/Feb/2019:11:43:35 +0000] "GET /parse?model=model_20190222-163549&project=ins&q=%E4%BD%A0%E5%A5%BD HTTP/1.1" 200 925 "-" "python-requests/2.21.0"
but when I use POST request to test the rasa-nlu from command line ,it works well:
$ curl -XPOST localhost:5000/parse -d '{"q":"你好", "project": "ins","model":"model_20190222-160808"}'
the response is correct:
{ "intent": {   "name": "hi",   "confidence": 0.7992279529571533 }, "entities": [], "intent_ranking": [   {     "name": "hi",     "confidence": 0.7992279529571533   }, ...
It seems like that the problem is in rasa-core,but I am not sure.

Anybody could help me. Forgive me the rude question. Many thanks!!
@tmbo

rasa-nlu's log is like this : it indicate the the rasa-core uses a GET request
2019-02-22 19:43:36+0800 [-] "<IP_ADDRESS>" - - [22/Feb/2019:11:43:35 +0000] "GET /parse?model=model_20190222-163549&project=ins&q=%E4%BD%A0%E5%A5%BD HTTP/1.1" 200 925 "-" "python-requests/2.21.0"
but when I use POST request to test the rasa-nlu from command line ,it works well:
$ curl -XPOST localhost:5000/parse -d '{"q":"你好", "project": "ins","model":"model_20190222-160808"}'
the response is correct:
{
"intent": {
"name": "hi",
"confidence": 0.7992279529571533
},
"entities": [],
"intent_ranking": [
{
"name": "hi",
"confidence": 0.7992279529571533
},

Forgive me the rude question. Many thanks

Thanks for raising this issue, @wochinge  will get back to you about it soon.

@suenpun Can you please attach

the content of your endpoint configuration file
your startup command for rasa core
the stack trace of the error

Thank u so much! I've changed the charset of nlu training json-data to utf-8 . It works ,but I'm not sure whether this is the source of the problem. If there are any further questions. I'll get back to you later. Thank u again! @akelad @wochinge

Sounds great! If you encounter further issues, please create a new one. I'll close this one for now


Skipping DASH manifest: ExtractorError

What I get:
"WARNING: [youtube] yOaoUx0Mc0Q: Skipping DASH manifest: ExtractorError(u'Could not download DASH manifest: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)"
····
~$ youtube-dl -U
youtube-dl is up-to-date (2015.07.21)

Duplicate of #5118, it will be fixed in the next version.
For future reports, please post the full output a explained in the error message.


Preinitialize MountPoints to avoid assigning to a nil map

Fixes #13435

:+1:

As we discuss in #13448 , this is the proper way +1:

Seems like there are a couple of other places where we have this problem.

LGTM.

Seems like there are a couple of other places where we have this problem.

@cpuguy83 can you point to those other places? So far the only references to &Container{} that I find are the one in this patch and the one in newContainer.

I was thinking in registerMountPoints, but it seems like these maps are ok.

LGTM


Fix compilation errors on unsupported platforms

将ruapu_some_inst的定义统一为typedef void (*ruapu_some_inst)()，而非每个平台都去重新定义一次，这样会导致不支持的平台（比如Emscripten）缺少ruapu_some_inst的定义从而无法编译。
string.h头文件也统一include，因为对外接口函数ruapu_supports使用了strcmp, 同时还使用了类型size_t，这同样修复了不支持的平台因缺少头文件而编译失败。
同时由于在 #98 中将MSVC编译器检测异常的方式由VEH改为SEH，ruapu在UWP下也可以工作了，因此删除了单独检测UWP的代码。
UWP下MSVC编译器禁止从unsigned char[] 到 void (*)()类型的转换，这可能是之前将windows平台下的ruapu_some_inst类型定义为const void*的原因，实际上MSVC在编译普通C/C++时并无此限制。这里采用一种兼容性更好的写法，将指令先转为void *再转为void (*)()，解决了此问题。
修改主要可能影响Windows平台，因此我对window平台进行了测试，主要编译运行main.c文件：
Mingw64: gcc和clang，不加优化参数和开-O3编译皆正常
MSVC: 普通C++和UWP，开Debug或者Release编译皆正常

Define ruapu_some_inst as typedef void (* ruapu_some_inst) only once instead of redefining it every platform, which may result in unsupported platforms (such as Emscripten) missing the definition of ruapu_some_inst and unable to compile.
The string. h header file is also include only once because the interface function ruapu_supports uses strcmp and size_t. This also fixes the compilation error of  lack of header file on unsupported platforms.
At the same time, due to changing the way the MSVC compiler detects exceptions from VEH to SEH in #98 , ruapu can now work under UWP, so the code for detecting UWP has been removed.
Under UWP, the MSVC compiler prohibits type conversion from unsigned char [] to void (*) (). This may be the reason of using const void* instead of void (*)() as the type of ruapu_some_inst on the Windows platform. In fact, MSVC does not have this restriction when compiling normal C/C++. A more compatible method is adopted here, which first converts the instruction to type void * and then to type void (*) () to solve this problem.
The modifications mainly affect the Windows platform, so I tested it, mainly compiled and ran the main. c file:
Mingw64: gcc and clang, built without optimization parameters and with - O3, all OK.
MSVC: Normal C++and UWP, built with Debug and with Release, all OK.

Thanks for your contribution !


Disable Practice button when Explorer Practice is on.

Especially on mobiles, it's so easy to click on Practice instead of go to start.

Implemented in v2.3.70


Look into simplifying formatters

The current formatter code is kind of yucky, and has several minor faults:

Formatter immune stuff doesn't play nicely with history - it invokes format_phrase multiple times instead of just once, creating multiple history entries.
The way formatters are represented is just kinda ugly.
Unformatting doesn't handle snake_case.

Andreas Arvidsson completely rewrote them (https://github.com/AndreasArvidsson/andreas-talon/blob/master/text/text.talon, https://github.com/AndreasArvidsson/andreas-talon/blob/master/text/formatters.py). Perhaps something along those lines would be better?

Some concerns about formatter behaviour that are relevant: #539 #674 #616 #539

Addressed by https://github.com/talonhub/community/pull/1431


Express most built-ins as macros

We're happy when most built-in functions can be expressed as "pure 007", as opposed to extranatural substrate (currently Perl 6, in general whatever). There's a tendency to explain as many built-ins as possible within the system being defined, 007.
So also for terms, operators, and statements. This issue tracks that.
Some things are really simple, such as infix:<+>. No problem; it just maps to an op in the VM layer. Ditto for many other operators.
Some things are tricky or impossible to define without running into circularity issues. Expressions as such. Function calls. Function declarations. In some of these cases we might get pretty far, in the sense of seeming to provide an in-system explanation. But in another sense we probably have to give up and breathe metacircular magic into those things in order for them to work.

By the way, @raiph, this is very close to what I once referred to as an Algoloid that uses macros to define itself.
In a way, it's nothing exciting. Just a bigger metacircular loop than Lisp typically has. One could define most of the core statements of 007 in terms of each other, or bottoming out as instructions on the VM layer. Instructions on the VM layer could then be interpreted by an interpreter written in pure 007. The whole metacircular loop would bottom out by having a VM written in some portable language, such as C.


First Run issues

Thanks in advanced.
How do I connect to mysql/Maria on first run?
Is there a default username and password?
do I need to setup the database myself?
It would be nice to actually document this in the README. Clear instructions seem to be an issue with these docker containers. If the info is somewhere else perhaps a simple copy and paste is not asking too much?
Thank you!

Hello
We would recommend to use Nextcloud instructions section where you can find all necessary options for configuring your containers.
For example here you can find some examples of the docker-compose files


Documentation and package.json should list minimum Node version as 16

This package will not work with Node14 for example because it uses FileHandle.createReadStream, which doesn't exists in Node14.  Adding the minimum Node version to the docs and listing in the engines config in package.json will save people a lot of headache wondering why things fail out of the box when using versions pre-16

Thanks for pointing this out. I'll make the update soon.

I released this with v1.2.2. I went with node 18 because of https://github.com/hyperjump-io/json-schema/issues/24. 18 is the current LTS version, so hopefully that's reasonable.

Interestingly enough there is a lot of stuff thats still using Node 14 and/or 16.  GCP for example.

Oh, I know. But, since this is a new library, I'm not concerned about breaking legacy code. I'm also not interested in catering to people still using EOL versions of node. That's a security risk and I'd rather not enable bad choices. So, node 14 support is out of the question. Node 16 is still maintained, but I would like to migrate to the built-in version of fetch soon rather than using the node_fetch dependency. I can only do that with node 18, so while there's nothing wrong with using node 16 right now, it's more important to me to be able to use a node 18 feature than it is to support people still on node 16.


Pk2D and Tk3D in v3

All done, back to you @nikfilippas

@damonge as discussed on Slack, I added the option to initialize the Pk2D object from spline_parameters. Internally, all it really needs is something that has the __get__ method (i.e. the dot) to access the spline parameters. So, this will work with both the C-level API, and the Python API, which is nice.
Users can either pass cosmo.cosmo.spline_params (if they prefer the old way) or an instance of ccl.parameters.SplineParams, (such as ccl.spline_params).

Back to you @nikfilippas

@damonge ok one final review comment remaining, and it's done!

Back to you @nikfilippas


Factor update method

Makes webkit's JIT compiler have easier time optimising this function.

@g-zachar LGTM, do you think need to run specific tests or can add this as is?

@woutermeek If browser team confirms that it indeed solves JIT compiler issue or at least improves on it then we can merge.


Constraint version pydantic to ~=1.8.2 (#3585)

Pull request opened by the merge tool on behalf of #3585

Merged into branches iso4 in 3d30c9089b8c81105a94e900833b309f98b8b212


[Storage][DataMovement] Updated perf test readme, remove perf from base sln

Just some cleanup from perf tests.

API change check
API changes are not detected in this pull request.


Exception: Insufficient Base Funds! (Actual: 0.0, Minimum: 0.0001)

Describe the bug
I have made a SELL manually
Exception: Insufficient Base Funds! (Actual: 0.0, Minimum: 0.0001)
after git pull and relaunching the Bot
getting : Exception: Insufficient Base Funds! (Actual: 0.0, Minimum: 0.0001)

problem resolved  after executing the BOT with  : python3.9 pycryptobot.py --lastaction SELL

I'm running into a similar issue:

After I ran the "...--lastaction SELL" I got this error message:

I think this is related to other issues but I'm not sure how or what to do to fix this.
I'm attempting to start a bot for the first time using Coinbase Pro for DOT.


open-vm-tools doesn't enable display autoscaling on Wayland

Issue description
When you set services.vmwareGuest.enable = true on NixOS and then start a wayland display manager (in my case - sway) display autoscaling should be enabled (i.e. when you resize the VM window on VMWare it should stretch the display resolution out.)
That doesn't happen and when you try to run the vmware-user-suid-wrapper or vmware-user and instead you get the following error:
Xlib:  extension "VMWARE_CTRL" missing on display ":0".

Steps to reproduce
1.) Enable services.vmwareGuest.enable = true;
2.) Rebuild nixos-rebuild switch
3.) Start sway (or alternative Wayland-Based DM)
4.) Start vmware-user-suid-wrapper
5.) See the error above.
Technical details
 - system: `"x86_64-linux"`
 - host os: `Linux 4.17.15, NixOS, 18.09pre149415.8395f9aa85e (Jellyfish)`
 - multi-user?: `yes`

Is this implemented for wayland even? xlib is x11 specific.

Too me this sounds more like an upstream issue.

You should @Mic92 Ok so yes they do "kinda" have Wayland support (they announce it as Wayland support but really it's Xwayland support.) I believe what's happening here is:

vmware.service needs access to udev in order to load the resolutionSet.so and resolutionKMS.so plugins respectively.
It currently does not have this in the "environment" part of the service file so I guess I need to figure out how to add that!

I think simply adding dbus to the above will enable the plugin to load and the issue to be resolved. I think that plugin is not needed for Xorg but is needed for Wayland support.

@cawilliamson did the LD_LIBRARY_PATH suggestion help?

I'm experiencing the same issue with NixOS vmware guest running Gnome 3 under Wayland. I can reproduce what @cawilliamson described.

In comparison: My Ubuntu vmware guest runs Gnome under Wayland and doesn't have any problem scaling the screen (as long as libgtkmm3 is installed, which was a problem earlier. But I can see that it is in the closure of the nix open-vm-tools package).
I can confirm that it is an issue with Wayland: When I start Gnome with X.org the display resizes correctly. (But then the mouse stops working for some reason)

@peterhoeg I was told on the IRC to ping you since you have worked on this NixOS module.

I switched channels to 20.09-beta and noticed something curious:
The screen scaling bug still occurs, but only when I have displayManager.gdm.enable = true;!
@hedning, @jtojnar, @worldofpeace, @dasj19, this seems to be an issue with gdm.

I can reproduce this on 20.09 but unfortunately if I disable wayland the mouse stops working for non-root users and gdm.

This gets around the display issue and somewhat fixes the mouse though it will won't move out of the guest window and is a little choppy.
# Fixup screen scaling in wayland for open-vm-tools
  nixpkgs.overlays = [
      (self: super: with super; {
        open-vm-tools = open-vm-tools.overrideAttrs (old: {
          postFixup = old.postFixup or "" + ''
            l=$out/lib/open-vm-tools/plugins/vmsvc/libresolutionKMS.so
            old_rpath="$(patchelf --print-rpath $l)"
            patchelf --set-rpath "$old_rpath:${systemd}/lib:${libdrm}/lib" $l
            l=$out/lib/open-vm-tools/plugins/vmusr/libresolutionSet.so
            old_rpath="$(patchelf --print-rpath $l)"
            patchelf --set-rpath "$old_rpath:${systemd}/lib:${libdrm}/lib" $l
            l=$out/lib/open-vm-tools/plugins/common/libvix.so
            old_rpath="$(patchelf --print-rpath $l)"
            patchelf --set-rpath "$old_rpath:${linux-pam}/lib" $l
          '';
        });
      })
    ];

    environment.etc."vmware-tools/tools.conf" = {
      mode = "0444";
      text = ''
        [resolutionKMS]
        enable=true
    ''; };

Might be useful to have this fix in the actual nixpkgs than.

To fix this issue also https://github.com/NixOS/nixpkgs/pull/138754 is required. If you are still on 21.05 please upgrade to 21.11 or pick the commit.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/help-needed-to-make-sway-and-wayland-work/20490/4

Doesn't seem to work for me (image stretched instead of resolution being updated). I have 3D acceleration enabled and use greetd, not sure if they matter.
My configuration.nix:
{ config, pkgs, ... }:

{
  imports = [ ./hardware-configuration.nix ];
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;
  virtualisation.vmware.guest.enable = true;
  networking.useDHCP = false;
  networking.interfaces.ens33.useDHCP = true;

  programs.sway.enable = true;
  services.greetd = let
    swayRunner = pkgs.writeShellScript "sway" ''
      export XDG_SESSION_TYPE=wayland
      export XDG_SESSION_DESKTOP=sway
      export XDG_CURRENT_DESKTOP=sway
      export WLR_NO_HARDWARE_CURSORS=1
      ${pkgs.sway}/bin/sway
    '';
  in {
    enable = true;
    settings = rec {
      initial_session = {
        command = swayRunner;
        user = "name";
      };
      default_session = initial_session;
    };
  };

  users.mutableUsers = false;
  users.users.name = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
    hashedPassword = "<password>";
  };

  system.stateVersion = "22.11";
}

nix-channel --list
nixos https://nixos.org/channels/nixos-unstable


Add additional parameters to params

I have two fields, most of the requests are required, but I don't want to repeat these two fields in all params. Is there any way to solve them?
Sorry, my English is not very good, use Google translate, if you can solve it, thank you very much

It would be great if I can see the pseudo code :)

    /// encoding + parameters.     var params: Parameters? {         switch self {         case .Info():             return ["userId": AccessToken.current?.userId ?? "",                     "token": AccessToken.current?.token ?? ""             ]         case .ChangeUsername():             return ["UserId": AccessToken.current?.userId ?? "",                     "token": AccessToken.current?.token ?? ""             ]         default:             return nil         }     }
For example, ".Info" and ".ChangeUsername" both require "userId" and "token" values, including more requests in the future also need these two fields, so I do not want to do repetitive work, I try using plugin Are added to the "params" before making the request, but "params" is read-only. Is there a solution, thank you very much

/// encoding + parameters.
var params: Parameters? {
   switch self {
        case .Info():
            return ["userId": AccessToken.current?.userId ?? "",
                    "token": AccessToken.current?.token ?? ""
            ]
       case .ChangeUsername():
            return ["userId": AccessToken.current?.userId ?? "",
                    "token": AccessToken.current?.token ?? ""
            ]
        default:
           return nil
     }
 }

I think you can use as following:
var params: Parameters? {
  var parameters: [String: Any] = [
    "userId": AccessToken.current?.userId ?? "",
    "token": AccessToken.current?.token ?? "",
  ]
  switch self {
  case .Info():
    parameters["additionalKey1"] = additionalValue1
    return URLEncoding() => parameters

  case .ChangeUsername():
    parameters["additionalKey2"] = additionalValue2
    return URLEncoding() => parameters

  default:
    return nil
  }
}

PS: I re-formatted your comment and deleted the duplicate :)

Thank you for your help  ！


added python batch example; updated python catalog example; removed t…

…enant URL; shipping required for DE

Docs Build status updates of commit 95bc4cd:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Advertising/shopping-content/catalogs-resource.md
:white_check_mark:Succeeded

Advertising/shopping-content/code-example-create-batch-request.md
:white_check_mark:Succeeded

Advertising/shopping-content/code-example-manage-catalogs.md
:white_check_mark:Succeeded

Advertising/shopping-content/get-started.md
:white_check_mark:Succeeded

Advertising/shopping-content/how-get-status-product-offers.md
:white_check_mark:Succeeded

Advertising/shopping-content/inventory-resource.md
:white_check_mark:Succeeded

Advertising/shopping-content/manage-catalogs.md
:white_check_mark:Succeeded

Advertising/shopping-content/manage-products.md
:white_check_mark:Succeeded

Advertising/shopping-content/products-resource.md
:white_check_mark:Succeeded

Advertising/shopping-content/status-resource.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 735f71f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Advertising/shopping-content/catalogs-resource.md
:white_check_mark:Succeeded

Advertising/shopping-content/code-example-create-batch-request.md
:white_check_mark:Succeeded

Advertising/shopping-content/code-example-manage-catalogs.md
:white_check_mark:Succeeded

Advertising/shopping-content/get-started.md
:white_check_mark:Succeeded

Advertising/shopping-content/how-get-status-product-offers.md
:white_check_mark:Succeeded

Advertising/shopping-content/inventory-resource.md
:white_check_mark:Succeeded

Advertising/shopping-content/manage-catalogs.md
:white_check_mark:Succeeded

Advertising/shopping-content/manage-products.md
:white_check_mark:Succeeded

Advertising/shopping-content/products-resource.md
:white_check_mark:Succeeded

Advertising/shopping-content/status-resource.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Cherrypicks for 1.5

182804819 not found in github

@gunan any idea of the fix for presubmits?

Could you also cherrypick 725d049caea2b75eae373760127ed9b55138f7dc

@yifeif can I merge this PR, or is a fix coming soon?

Re-triggering

Looks like a flaky test?
[ RUN      ] FileBlockCacheTest.CoalesceConcurrentReads
tensorflow/core/platform/cloud/file_block_cache_test.cc:464: Failure
Value of: WaitForNotificationWithTimeout(&notification, 10000)
  Actual: false
Expected: true
Timeout waiting for concurrent thread to start.
[  FAILED  ] FileBlockCacheTest.CoalesceConcurrentReads (110 ms)


fix: fix incorrect proto name where javapackage or protopackage is empty

DescriptorMap contains incorrect protoname in case that protopackage or javapackage is empty string.
For example:
If protopackage is empty string, typename becomes ..ProtoMessage when it should be .ProtoMessage because we are constructing it as String.format(".%s.%s", protoPackage, className)
In this fix, we are not using double dots in case that protopackage is empty string.

LGTM 👍
Just a small thing, can you remove the second redundant "fix" in the commit message? That is fix: fix.


Allow a summary to be specified for widget pages

Purpose
Previously only pages used the abstract parameter to populate the description tag. It was not possible to set the description tag on widget sites.
This change fixes that.

Thanks for your contribution @dschuermann :)
For others reading this, I recommend defining summary = "your page summary" in  a widget page's _index.md rather than using abstract as we may consider converting all abstract options to summary in the future to unify these parameters.


Android Trace Replay of Multiple Window Surfaces Fail

Trying to replay a captured PUBG trace on the capturing platform (Pixel4) results in native window attachment failure. The relevant logcat snippet:
07-21 07:45:45.876   600  1909 E BufferQueueProducer: [com.lunarg.gfxreconstruct.replay/android.app.NativeActivity#0] connect: already connected (cur=1 req=1)
07-21 07:45:45.876 14339 14362 E vulkan  : native_window_api_connect() failed: Invalid argument (-22)
07-21 07:45:45.876 14339 14362 F gfxrecon: API call vkCreateAndroidSurfaceKHR returned error value VK_ERROR_NATIVE_WINDOW_IN_USE_KHR that does not match the result from the capture file: VK_SUCCESS.  Replay cannot continue.
07-21 07:45:45.908   824   850 D rlsservice: MonitorDisplayStatus client num 1, display status 1, sensor status 1
07-21 07:45:45.914 14339 14362 V gfxrecon: Replay failed with error message: the requested window is already connected to another VkSurfaceKHR object, or some other non-Vulkan surface object

Is there a way to handle multiple android surfaces in GFXReconstruct? Perhaps an option to select only one to view during replay (with other commands dropped)?

We currently can't handle multiple surfaces for Android replay.  I'll look at adding support for this.

To clarify, the support you'll look at adding is to enable multiple surfaces or to select only one to render?

I'll look into both options.  Ultimately we want to be able to support multiple surfaces, but an option to select only one to render may serve as an intermediate solution that could be implemented quickly.
I think it would be possible to add an option that designates one surface to be treated as a native window surface, converting the other surfaces to headless surfaces, with minimal changes.
We can also try an intermediate solution where we create and reuse a single surface/swapchain for Android.


Reverse-engineer Pimoroni "synth" example with GDB

Goal is to translate this to zeptoforth

Happening in https://github.com/AlgoCompSynth/rp2040-forth-builder/issues/4

Official Windows toolchain build is working. I've forked the pimoroni-pico repo to https://github.com/AlgoCompSynth/pimoroni-pico so I can add print statements and other stuff I want in the Forth version. VSCode debugger works with the debug probe on the serial "Hello world" and I don't care whether the LEDs blink or not! Next step - see what the Pico sounds like.

Demo works - kind of a chiptune groovebox sound

The core C++ code is in repository https://github.com/raspberrypi/pico-extras/tree/master/src/rp2_common/pico_audio_i2s. This is what the Pimoroni Audio Pack demo uses. It's in C, manages buffers, uses a PIO and is frightfully complicated. As near as I can tell it's hard-coded to 16 bits. It's probably too much for a proof of concept.
The DAC on the Pimoroni Pico Audio Pack can go up to 384 kHz and 32 bits, although it's not clear there's any gain in audio quality over 24 bits. I think I need to build something simpler than pico_audio_i2s.

The Cornell lab code appears to be 3-clause BSD licensed! So we need the I2S code from pico_extras but we can use the DDS code from Cornell, for example https://github.com/vha3/Hunter-Adams-RP2040-Demos/blob/master/Lab_1_Incremental/b_Timer_Interrupt_DDS_Demo/dactest.c

The demos as written are very complex. I don't believe all that complexity is necessary given that we have a real-time OS!


chore(sdk): define helper trait BlockPrimitives

Based on https://github.com/paradigmxyz/reth/pull/12647

Fixes bug in stages. Bug: writing compact encoded signed transaction instead of the unsigned transaction to db
Defines helper trait BlockPrimitives, which links NodePrimitives::Block to NodePrimitives::BlockHeader,  NodePrimitives::BlockBody and NodePrimitives::SignedTransaction. This makes the trait bounds necessary on impl bodies much less verbose!
Makes NodePrimitives super trait of FullNodePrimitives, as it allows for the transition FullNodePrimitives -> BlockPrimitives -> NodePrimitives.
NOTE: the reason I didn't do this in the first place is because I copied the pattern of FullNodeComponents -> RpcNodeCore used to simplify type definitions, introduced because FullNodeComponents wasn't built from a base trait without trait bounds (and re-writing that would have been a too big change for the task). Here we have the opportunity to have a plain (trait unbound) super trait though since we're writing primitives from scratch.

Fixes bug in stages. Bug: writing compact encoded signed transaction instead of the unsigned transaction to db

I believe we've been always writing signed transaction to both staticfiles and database?
https://github.com/paradigmxyz/reth/blob/166a2346dcd2a9c8d5fea6995a9fc5e6426c62bb/crates/storage/provider/src/providers/static_file/writer.rs#L543-L547
https://github.com/paradigmxyz/reth/blob/166a2346dcd2a9c8d5fea6995a9fc5e6426c62bb/crates/storage/db/src/tables/mod.rs#L295

Fixes bug in stages. Bug: writing compact encoded signed transaction instead of the unsigned transaction to db

I believe we've been always writing signed transaction to both static files and database?
https://github.com/paradigmxyz/reth/blob/166a2346dcd2a9c8d5fea6995a9fc5e6426c62bb/crates/storage/provider/src/providers/static_file/writer.rs#L543-L547
https://github.com/paradigmxyz/reth/blob/166a2346dcd2a9c8d5fea6995a9fc5e6426c62bb/crates/storage/db/src/tables/mod.rs#L295

TransactionSigned doesn't impl Compact though

It does since https://github.com/paradigmxyz/reth/pull/12539
related discussion https://github.com/paradigmxyz/reth/pull/12694#discussion_r1851075736

then it should impl Compact, but it doesn't. before StaticFileProviderRW::append_transaction took TransactionSignedNoHash. @klkvr

then it should impl Compact, but it doesn't

It does impl Compact https://github.com/paradigmxyz/reth/blob/e3702cfc87449294da061f02a571a23061b8ab50/crates/primitives/src/transaction/mod.rs#L1054

before StaticFileProviderRW::append_transaction took TransactionSignedNoHash

yep, but now it takes anything that implements Compact (should probably change that to N::SignedTx given that we have primitives on StaticFileProvider now)
sorry for the confusion with the 2 storage types, I was expecting the TransactionSignedNoHash to be gone by now

ok nvm, switching between a lot of branches to shrink scope, my bad

re-request review @klkvr

since we won't have access to a base trait without trait bounds on associated types since https://github.com/paradigmxyz/reth/pull/12846, there is no need for this pr


Change Port from 8080

Is it possible to change the port it runs on from 8080 to something else?

Yes, you can by altering the port number on line 26 in ./config/index.js.

@flxwu it worked. Thank You. Now I keep getting error code 3..
Compiled successfully in 67036ms                              11:07:15 PM

Listening at http://localhost:8081

(node:25540) UnhandledPromiseRejectionWarning: Error: Exited with code 3
at ChildProcess. (/home/stackedit/node_modules/opn/index.js:83:13)
at Object.onceWrapper (events.js:317:30)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Socket.stream.socket.on (internal/child_process.js:346:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Pipe._handle.close [as _onclose] (net.js:567:12)
(node:25540) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:25540) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.


Remove everything related to the database

@nichtich Can we remove anything related to database access, or do you think it'll be necessary to use a database again in the future?

I think we will use a database in the future for #29 and #36 but these will better be another kind of database. Actually I just started with SQLite as simple cache to not call coli-ana twice with same notation. For more sophisticated queries we should better experiment with Neo4J and/or Solr to include full DDC hierarchy, DDC analysis and K10plus titles indexed with DDC. In any case this will first be an independent repository that uses coli-ana as backend (and/or the reverse) but not part of coli-ana.

Alright, sounds reasonable. 👍

@nichtich Could you check the commit where I removed the database-related stuff? The only things I left were a reference in the README (with a note that the database backend is not used anymore) as well as the error type - although I added a TODO there that we should add proper error types that are used throughout the application.
In particular, there was still a reference to GET /analyze?member=members which I could not find in the code. Was this removed before, when we decided to remove the database backend?