Programming exercises: Add possibility to configure docker run arguments on bamboo via spring properties

Checklist
General

[x] I tested all changes and their related features with all corresponding user types on a test server.
(https://docs.artemis.ase.in.tum.de/dev/guidelines/language-guidelines/).
[x] I chose a title conforming to the naming conventions for pull requests.

Server

[x] I followed the coding and design guidelines.
[x] I implemented the changes with a good performance and prevented too many database calls.
[x] I documented the Java code using JavaDoc style.

Motivation and Context

Programming Exercises may negatively influence the Build Infrastructure if not prevented explicitly. Especially C programming exercises can crash Build Agents with memory leaks or fork bombs. Since the exercises are executed within a docker container, we can limit the containers resources explicitly and kill the container before the agent crashes.
Description

This change only impacts bamboo based deployments at the moment

In this PR we add a new configuration option for docker run arguments: artemis.continuous-integration.build.default-docker-flags. An Administrator may provide any docker run flag as a key-value pair to this config. Artemis will explicitly add those flags to all newly created Bamboo Build plans.
We chose a default limit of 2 CPUs, 2GB Ram and 1000 pids. These values should be sufficient for most of the exercises. If they are not, instructors have to adapt the base and template build plan in bamboo with more resources (and respective bamboo "requirements" to only use more powerful agents).
Acknowledgements:

@sleiss: Helping with Spring magic
@RobertJndw and @kliwniloc: Help debugging the build infrastructure

Steps for Testing

Prerequisites:

1 Instructor

Steps in Artemis

Log in to Artemis
Navigate to Course Administration
Select a Course
Create a new Programming Exercise
Open the base and solution build plans in bamboo (links can be found in the lower half of the exercise overview page)

Steps in Bamboo

Click on "Actions" and then on "Configure plan" in the top right corner of the build plan
Select the "Default Job"
Change to the "Docker" Tab
Inspect the "Extra arguments". They should look like this:

--cpus=2
--memory=2g
--pids-limit=1000
--memory-swap=2g

Check that both build plans build successfully

Review Progress

Code Review

[ ] Review 1
[ ] Review 2

Manual Tests

[x] Test 1
[ ] Test 2

Exam Mode Test

[ ] Test

Thank you for testing!
The docker resource management is unfortunately not really intuitive... If you set a memory limit of e.g. 2GB, the container can use 2GB of RAM + the same amount of SWAP = 4GB of RAM before it gets killed.
The documentation states:

If --memory-swap is unset, and --memory is set, the container can use as much swap as the --memory setting, if the host container has swap memory configured. For instance, if --memory="300m" and --memory-swap is not set, the container can use 600m in total of memory and swap.

Now that I think about it, the best solution seems to be to set the --memory-swap flag to 2GB as well since our agents only have 4GB of RAM. Thanks for pointing that out!

@tunargs Thanks for testing - the extra argument is the result of previous feedback. I forgot to add it in the description. Thanks for pointing that out!


discord.js v13.4.0 depreciation warning

Please describe your problem in as much detail as possible:
DeprecationWarning: Channel#deleted is deprecated, see https://github.com/discordjs/discord.js/issues/7091.
at VoiceChannel.get deleted [as deleted] (/app/node_modules/discord.js/src/structures/Channel.js:76:15)
at isSupportedVoiceChannel (/app/node_modules/distube/dist/util.js:155:17)
at DisTube.playVoiceChannel (/app/node_modules/distube/dist/DisTube.js:144:45)
at DisTube.play (/app/node_modules/distube/dist/DisTube.js:119:20)
Further information:
It's working fine, but whenever I start the bot for first time I'm getting the above error. Except that there is no compatible issue with djs v13.4.0

DisTube version:
discord.js version: 13.4.0
Node.js version: 17.x
Operating system: linux

Just deprecated update


Data-driven tests are not fully supported for overloaded test methods

Description
The MSTest V2 test framework does not appear to correctly support overloaded data-driven test methods.
Only tests for the first overload are actually found and executed.
Steps to reproduce
Compile and execute the unit tests in the following code fragment:
using System.Globalization;

namespace TestProject1
{
    [TestClass]
    public class UnitTest1
    {
        [DataTestMethod]
        [DynamicData(nameof(Equal_StringAndInt32), DynamicDataSourceType.Method)]
        public void Equal(string x, int y)
        {
            Assert.AreEqual(x, y.ToString(CultureInfo.InvariantCulture));
        }

        [DataTestMethod]
        [DynamicData(nameof(Equal_Int32AndString), DynamicDataSourceType.Method)]
        public void Equal(int x, string y)
        {
            Assert.AreEqual(x.ToString(CultureInfo.InvariantCulture), y);
        }

        private static IEnumerable<object[]> Equal_StringAndInt32()
        {
            yield return new object[] {"1", 1};
            yield return new object[] {"2", 1};
        }

        private static IEnumerable<object[]> Equal_Int32AndString()
        {
            yield return new object[] {1, "0"};
            yield return new object[] {2, "2"};
        }
    }
}

Expected behavior
The test results in both Visual Studio and the console runner report 4 tests: two for the Equal(int x, string x) overload and two for the Equal(string x, int x) overload. For each of these overloads, one test should pass, and one should fail.
Actual behavior
Only the results for the Equal(int x, string x) overload are reported in Test Explorer:

Running dotnet test -t --nologo --no-restore --no-build also finds only two tests:
Test run for TestProject1.dll (.NETCoreApp,Version=v6.0)
The following Tests are available:
    Equal (1,1)
    Equal (2,1)

Running dotnet test --nologo --no-restore --no-build only executes two tests:
Test run for TestProject1.dll (.NETCoreApp,Version=v6.0)
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Failed Equal (2,1) [13 ms]
  Error Message:
   Assert.AreEqual failed. Expected:<2>. Actual:<1>.
  Stack Trace:
     at TestProject1.UnitTest1.Equal(String x, Int32 y) in C:\Users\exg906\source\repos\TestProject1\UnitTest1.cs:line 12

Failed!  - Failed:     1, Passed:     1, Skipped:     0, Total:     2, Duration: 32 ms - TestProject1.dll (net6.0)

Environment

Operating system: Windows 10
MSTest framework and adapter: v2.2.10
Visual Studio Enterprise 2022 (17.3.3)
Target framework: .NET 6.0
Test Execution Command Line Tool: v17.3.0
Microsoft.NET.Test.Sdk: v17.3.1

Hi @drieseng,
Thanks for reporting this!
I have just tested and can reproduce the issue you are seeing. I have also tested with DataRow and it's also failing. I will look into the design documents to see whether this is supposed to be a supported scenario (in which case we have a bug) or if this is something we should try to support (new feature/enhancement).

I cannot find any mention that supports the fact this is a supported feature and testing a few older versions I cannot see it ever working so I will consider this as a feature request!

@Evangelink, the fact that these are silenty ignored can already be considered ok. Perhaps there's no documentation that indicates it is supported, but there's also no document that says it isn't.
If support this overloaded methods is added, I suppose the signature (parameter types) of the overload should be included in the name of the test method.
For example:
The following Tests are available:
    Equal(String, Int32) (1,1)
    Equal(String, Int32) (1,1)
    Equal(Int32, String) (1,0)
    Equal(Int32, String) (2,2)

To decide whether you'd want to mention the type name with or without namespace, or - when available - the C# keyword.

I understand that the behavior of silently ignoring is not ideal and should be fixed but detecting and raising some warning for this case is going to take the same time as actually implementing this request.
Besides, I am not saying we cannot support it. Actually, it's the opposite and that's why I flagged this as new feature.
Depending on the progresses we make with the scope of what needs to be done for 2.3.0, we may add it there, otherwise for sure it will be part of our v3 roadmap.

@Evangelink, if you can point me in the right direction (and if you've decided how such overloaded test methods should be reported), I can try to take a stab at this.

I really appreciate that you are offering some help here! We are currently refactoring the way we produce the test ID because it is broken for datasources, I hope that he fix will either make this feature directly supported or easy to add.
Because of that, I would like to avoid having you working for nothing (or have to redo everything you did after our change). If that's ok with you, I'll ping you here once our fix is available so you can start working on it.

Hey @drieseng, test ID feature is merged and I am done with the first pass of big refactorings. Let me know if you are still interested and I will try to find the place where the change would be needed.

@Evangelink: sure, I'll be happy to help.

Awesome, I'll get back to you early next week (or during weekend) with where you could start!

Are there any updates on this issue?

I am not sure why I didn't get back to @drieseng so let me start by apologizing! As for the update, not really but I think that should not be too complex to add that support so I'll plan it for 3.6 (we are already loaded for 3.5).


Implement Azure IoT Hub GW <> NS backend

Good morning @brocaar, I would like to try working on this one as I need it for a PoC.
Is it fine by you ?
What did you have in mind?
So far what I have seen is :

There is no sdk provided in Rust to connect Azure.
AMQP protocol could be used to connect a Service Bus Namespace, however the support of partition may be tricky
If we use the Event hub instead of the bus we can use a Kafka client to fetch the telemetry (https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview)

So I was thinking of using an Event hub and the kafka client (found the idea here https://www.adrians-blog.com/articles/software/2021/12/08/azure-eventhubs-in-rust/ )
And you are already using the dependency.
I think I would also need to change the gateway-bridge to send information about the region as well.
Thanks

I'm still in the development of this, but I'm planning to add a Redis based gateway backend (next to the MQTT). Using Redis Streams we have buffering + it makes it possible to use any language to integrate with this (and thus we can use the Go SDK).

Do you mean telemetry will be sent over redis in replacement of mqtt or two connections one for telemetry over mqtt and one for config like region ?

It would be something like (Redis Stream):
[ChirpStack Gateway Bridge] <- MQTT -> [IoT Hub] <- Service Bus -> [ChirpStack Azure Bridge] <- Redis Streams -> [ChirpStack]
Or (MQTT):
[ChirpStack Gateway Bridge] <- MQTT -> [IoT Hub] <- Service Bus -> [ChirpStack Azure Bridge] <- MQTT -> [ChirpStack]
After thinking about this a bit longer, maybe going the MQTT route is simpler + it makes the debugging easier as you can subscribe to the gateway MQTT topics directly to see what is going on. Depending the MQTT broker you use, it should not be a bottleneck.
I might start working on this too in the next couple of days, as I would like to get support for the Azure IoT Hub resolved for some clients :-) Originally I was planning to start working on the GCP IoT Hub first (#6), but they are deprecating the GCP IoT Hub.

Thanks for the explanation.
I have a better understanding. The downside I see it that you still need to host a Mqtt somewhere with this architecture. It can be a lot of overhead for some small companies like us :)
However I understand because the integration directly with the azure system is not easy with rust.
Because we needed to validate we could do it here is the POC I have done :
https://github.com/mogaleaf/chirpstack/pull/1
it comes along with a none breaking change in the gateway bridge to get the region :
https://github.com/mogaleaf/chirpstack-gateway-bridge/pull/1
I'm still waiting on a review from one person of my team for chirpstack and was planning to open a PR afterward.
I have used the kafka client to stream (from the EventGrid, not the routing) and the amqp client to push frames.
It's my first development with Rust so probably a bit messy but I have managed to done some tests and it works so far.
Indeed gcp is out that's why we are looking for option...

@mogaleaf This looks great, something like this would really help us out

Not bad for a first Rust project :+1:

I have a better understanding. The downside I see it that you still need to host a Mqtt somewhere with this architecture. It can be a lot of overhead for some small companies like us :)

Depending your needs, you have a MQTT broker up and running with an apt install mosquitto. But you are correct that it requires an additional component in-between which can be an advantage or disadvantage. It can be an advantage if you want to migrate from or to Azure IoT Hub, as you can also connect your gateways directly over MQTT. The other advantage is that it keeps the dependencies of each backend out of the ChirpStack project (some SDKs pull in a lot of dependencies).

it comes along with a none breaking change in the gateway bridge to get the region :

Not sure if that would be needed? I would imagine a single IoT Hub per region. Each region configuration in ChirpStack has already its own region name configured.

Thanks, I think you re totally right, no need to have the region in the gateway, I guess either one hub per region or on event queue based on other properties.
I'll make this change.
For the change of architecture (having a Chirpstack Azure Bridge), the other downside I see is that you have to make it scale and redundancy  (the mqtt) and also configure the security around it. While azure bridge already provides all of those features. You basically go to two queues before reading the frame. For us for instance it would be too much of a burden as we are a very small team.
thanks again

Which is why initially I was thinking about a Redis based backend, as Redis is already a requirement to setup ChirpStack.
Let me gather some more feedback from other companies I'm working with and that are using the Azure IoT Hub.

I'm closing this issue as currently this is not high on my list of priorities currently.


chore: move errors/ to lib/formatted-errors/errorsf

Changes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#1245
#1244  👈
#1243
#1242
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @NathanFlurry and the rest of your teammates on  Graphite


Documentation Issue : Wrong package name

What kind an issue is this?

[ ] Bug report. If you’ve found a bug, please provide a code snippet or test to reproduce it below.
The easier it is to track down the bug, the faster it is solved.

Issue description
Description
Wrong package name provided in the documentation.
Steps to reproduce
Please change import statement defined in Spark block to "import org.elasticsearch.spark.rdd.api.java.JavaEsSpark; " .
Code:
Test/code snippet

Strack trace:
Stack trace goes here

Version Info
ES-Hadoop   :  All
ES          :  All

Feature description

This should be fixed now with https://github.com/elastic/elasticsearch-hadoop/commit/73169c369aa7a01bdca3f7666d8e5767a058cb59 and https://github.com/elastic/elasticsearch-hadoop/commit/3676ce0673fa609a4f853dfe349838db9106dab1


Outdated contribution documentation

The contribution guidelines mention to run a postgres database using docker. The file path of the docker-compose file metioned there is outdated because the file extension has been renamed from .yaml to .yml.
I will provide a pull request to fix this issue.

Scanned the latest commits and found that this bug is already fixed.


VMS: Compensate for compiler type incompatibility

The compiler says that 'unsigned long long' isn't the same as
'unsigned __int64'.  Sure, and considering that crypto/rand/rand_vms.c
is specific VMS only code, it's easy to just change the type to the
exact same as what's specified in the system headers.

This pull request is ready to merge

Merged
2e5cdbc18a1a26bfc817070a52689886fa0669c2 VMS: Compensate for compiler type incompatibility


Date formatting fails on some Android devices

I haven't been able to reproduce this myself, but I am getting reports from the wild that sometimes LocalDate.format() is returning an invalid result on some Android devices. (Using https://github.com/JakeWharton/ThreeTenABP.)
Here's what happens, as confirmed by the log call in this method.

On entry to this method, month is 2016-08.
startOfMonth is correctly set to 2016-08-01.
monthString gets set to 0000-00-00 which is wrong.

private Observable<Void> ensureDataForMonthExistsInternal(YearMonth month) {
    final LocalDate startOfMonth = month.atDay(1);
    final String    monthString  = startOfMonth.format(DateTimeFormatter.ISO_LOCAL_DATE);
    Timber.i("Calling ensureDataForMonthExists with month %s, as date %s, formatted %s", month, startOfMonth, monthString);

So far the issue seems to affect only Motorola devices running Android 6.0.
Does threetenbp actually use possibly-broken system APIs for date formatting? Or is it completely self-contained? Intuitively, my expectation is that using DateTimeFormatter.ISO_LOCAL_DATE should not require the locale or any other information to be read from the system.
This was originally reported as https://github.com/JakeWharton/ThreeTenABP/issues/33.

The code in DateTimeFormatterBuilder L2352 gets the value, but it just does this by querying the input temporal. So, I can see no reason why it would return zero. Without the ability to reproduce, there isn't much I can do here.

We are experiencing a very similar issue with org.threeten:threetenbp:1.3.1.
So far, we had one error report coming from Motorola Moto G3 running Android 6.0.0.
We are using the following (simplified for readability) code to display dates:
ZonedDateTime now = ZonedDateTime.now();
ZonedDateTime[] datesToDisplay = new ZonedDateTime[8];
for (int i = 0; i < datesToDisplay.length; i++) {
    datesToDisplay[i] = now.plusDays(i);
}

We are using the following code to get day-of-week (text):
datesToDisplay[i].format(DateTimeFormatter.ofPattern("EE"));

And the following code to get day-of-month (number) and month-of-year (text):
datesToDisplay[i].format(DateTimeFormatter.ofPattern("d MMM"));

Day-of-week and month-of-year values are formatted correctly.
Day-of-month always outputs 0.
Since day-of-week is displayed correctly, I assume the problem might be somewhere in day-of-month formatting:

We might try to get our hands on Moto G3 device to reproduce this issue.

My workaround for this is simply to call LocalDate.toString() instead of formatting with DateTimeFormatter.ISO_LOCAL_DATE.
@jodastephen Is this a safe workaround? Do you think that LocalDate.toString() will ever return the date in a different format from the current uuuu-MM-dd?

That workaround is safe,  the toString format will not change

Hi,
I was able to reproduce this on a Motorola running 6.0 randomly a few times (which makes me think its a race condition).
This was happening with either of these lines of code:
String isoTime = LocalDateTime.now().format(DateTimeFormatter.ISO_DATE_TIME);
String formatted = LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd MMM yyyy"));

It seems like it is a race condition though, because the crash doesn't happen when I attach the debugger, but happens when I just run the app.
It is interesting because running the following line first logs the date correctly:
Log.d(TAG, "onCreate: local date :" + LocalDateTime.now());

Logs:
MainActivity_Test: onCreate: local date :2016-09-05T12:03:37.201
The stack track below is when either one of the two calls above fail. It is also interesting because sometimes it was crashing with HourOfDay and sometimes with DayOfMonth:
FATAL EXCEPTION: main  Process: za.co.riggaroo.dummydates, PID: 20456
java.lang.RuntimeException: Unable to start activity ComponentInfo{za.co.riggaroo.dummydates/za.co.riggaroo.dummydates.MainActivity}: org.threeten.bp.DateTimeException: Field HourOfDay cannot be printed as the value<PHONE_NUMBER> exceeds the maximum print width of 2
   at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2426)
   at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2490)
   at android.app.ActivityThread.-wrap11(ActivityThread.java)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1354)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:148)
   at android.app.ActivityThread.main(ActivityThread.java:5443)
   at java.lang.reflect.Method.invoke(Native Method)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:728)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)
Caused by: org.threeten.bp.DateTimeException: Field HourOfDay cannot be printed as the value<PHONE_NUMBER> exceeds the maximum print width of 2
   at org.threeten.bp.format.DateTimeFormatterBuilder$NumberPrinterParser.print(DateTimeFormatterBuilder.java:2361)
   at org.threeten.bp.format.DateTimeFormatterBuilder$CompositePrinterParser.print(DateTimeFormatterBuilder.java:1995)
   at org.threeten.bp.format.DateTimeFormatterBuilder$CompositePrinterParser.print(DateTimeFormatterBuilder.java:1995)
   at org.threeten.bp.format.DateTimeFormatter.formatTo(DateTimeFormatter.java:1385)
   at org.threeten.bp.format.DateTimeFormatter.format(DateTimeFormatter.java:1359)
   at org.threeten.bp.chrono.ChronoLocalDateTime.format(ChronoLocalDateTime.java:263)
   at org.threeten.bp.LocalDateTime.format(LocalDateTime.java:1828)
   at za.co.riggaroo.dummydates.MainActivity.onCreate(MainActivity.java:24)
   at android.app.Activity.performCreate(Activity.java:6245)
   at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1130)
   at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2379)
   at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2490)
   at android.app.ActivityThread.-wrap11(ActivityThread.java)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1354)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:148)
   at android.app.ActivityThread.main(ActivityThread.java:5443)
   at java.lang.reflect.Method.invoke(Native Method)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:728)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)

Just connecting the dots (if it can be any helpful), I've reported the same issue there: http://stackoverflow.com/questions/35043000/motorola-devices-org-threeten-bp-datetimeexception-when-parsing-a-date-in-thre
I thought it was correctly reported in that other issue #39 which is now fixed. But at the end it doesn't look like the same problem. #39 is about ofEpochDay() whereas this one is around format().

Some more information that might be useful. Getting the values separately works, as soon as I call format, it breaks.
  LocalDateTime localDateTime = LocalDateTime.now();
        Log.d(TAG, "getDayOfMonth(): " + localDateTime.getDayOfMonth() + ". getYear(): " + localDateTime
                .getYear() + ". getMonth(): " + localDateTime.getMonth() + ". getMonthValue(): " + localDateTime
                .getMonthValue());

        Log.d(TAG, "LocalDateTime.now(): " + LocalDateTime.now());
        final TextView textViewDate = (TextView) findViewById(R.id.text_view_date);

        String isoTime = LocalDateTime.now().format(DateTimeFormatter.ISO_DATE_TIME);

Produces the following as a log:

09-05 13:56:13.421 2497-2497/za.co.riggaroo.dummydates D/MainActivity_Test: getDayOfMonth(): 5. getYear(): 2016. getMonth(): SEPTEMBER. getMonthValue(): 9
09-05 13:56:13.421 2497-2497/za.co.riggaroo.dummydates D/MainActivity_Test: LocalDateTime.now(): 2016-09-05T13:56:13.421
09-05 13:56:13.430 2497-2497/za.co.riggaroo.dummydates E/AndroidRuntime: FATAL EXCEPTION: main
                                                                         Process: za.co.riggaroo.dummydates, PID: 2497
                                                                         java.lang.RuntimeException: Unable to start activity ComponentInfo{za.co.riggaroo.dummydates/za.co.riggaroo.dummydates.MainActivity}: org.threeten.bp.DateTimeException: Field MonthOfYear cannot be printed as the value<PHONE_NUMBER> exceeds the maximum print width of 2
                                                                             at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2426)
                                                                             at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2490)
                                                                             at android.app.ActivityThread.-wrap11(ActivityThread.java)
                                                                             at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1354)
                                                                             at android.os.Handler.dispatchMessage(Handler.java:102)
                                                                             at android.os.Looper.loop(Looper.java:148)
                                                                             at android.app.ActivityThread.main(ActivityThread.java:5443)
                                                                             at java.lang.reflect.Method.invoke(Native Method)
                                                                             at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:728)
                                                                             at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)
                                                                          Caused by: org.threeten.bp.DateTimeException: Field MonthOfYear cannot be printed as the value<PHONE_NUMBER> exceeds the maximum print width of 2
                                                                             at org.threeten.bp.format.DateTimeFormatterBuilder$NumberPrinterParser.print(DateTimeFormatterBuilder.java:2361)
                                                                             at org.threeten.bp.format.DateTimeFormatterBuilder$CompositePrinterParser.print(DateTimeFormatterBuilder.java:1995)
                                                                             at org.threeten.bp.format.DateTimeFormatterBuilder$CompositePrinterParser.print(DateTimeFormatterBuilder.java:1995)
                                                                             at org.threeten.bp.format.DateTimeFormatterBuilder$CompositePrinterParser.print(DateTimeFormatterBuilder.java:1995)
                                                                             at org.threeten.bp.format.DateTimeFormatter.formatTo(DateTimeFormatter.java:1385)
                                                                             at org.threeten.bp.format.DateTimeFormatter.format(DateTimeFormatter.java:1359)
                                                                             at org.threeten.bp.chrono.ChronoLocalDateTime.format(ChronoLocalDateTime.java:263)
                                                                             at org.threeten.bp.LocalDateTime.format(LocalDateTime.java:1828)
                                                                             at za.co.riggaroo.dummydates.MainActivity.onCreate(MainActivity.java:32)
                                                                             at android.app.Activity.performCreate(Activity.java:6245)
                                                                             at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1130)
                                                                             at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2379)
                                                                             at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2490)
                                                                             at android.app.ActivityThread.-wrap11(ActivityThread.java)
                                                                             at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1354)
                                                                             at android.os.Handler.dispatchMessage(Handler.java:102)
                                                                             at android.os.Looper.loop(Looper.java:148)
                                                                             at android.app.ActivityThread.main(ActivityThread.java:5443)
                                                                             at java.lang.reflect.Method.invoke(Native Method)
                                                                             at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:728)
                                                                             at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)

I'm not sure what other information would be useful?

In your test case, you call LocalDateTime.now() multiple times. I suspect that if you use localDateTime.format(DateTimeFormatter.ISO_DATE_TIME); on the last line, you won't find a failure until you see the previous two lines fail as well.  ie. this answer is probably correct. (ie. the result from System.currentTimeMillis() is incorrect, but the library isn't correctly validating it).
You can ensure that the object is created correctly by writing your own implementation of Clock that ensures that the result of System.currentTimeMillis() is sensible before returning it. You would then use LocalDateTime.now(MyValidatingClock.INSTANCE) to create the date/time.

In your test case, you call LocalDateTime.now() multiple times. I suspect that if you use localDateTime.format(DateTimeFormatter.ISO_DATE_TIME); on the last line, you won't find a failure until you see the previous two lines fail as well.  ie. this answer is probably correct. (ie. the result from System.currentTimeMillis() is ridiculously incorrect, but the library isn't correctly validating against it).
You can ensure that the object is created correctly by writing your own implementation of Clock that ensures that the result of System.currentTimeMillis() is sensible before returning it. You would then use LocalDateTime.now(MyValidatingClock.INSTANCE) to create the date/time.

Hi @jodastephen
I realised that I should have done that in the first example, I have updated the previous comment (I think a bit prior to your latest comment).
The issue still persists when using the same instance of LocalDateTime. I can correctly get all the information and only when running format() the exception does get thrown.
It looks like it is caused by app installation + race condition though which is strange. 🤔😮

I am also using version 1.3.2 where I see the Epoch issue might have been fixed? So it is possibly not related to #39

You may need to compile ThreeTen-Backport yourself with some extra debug statements in the code. Fundamentally, the formatter just uses this code which calls this code which is a simple switch statement, so if the year/month/day is correct when printed separately, there is probably something wrong with the JVM implementation on that phone.

I figured I'd add some information about my own experience with this issue.
We're getting user reports of zeroes appearing instead of expected values. That is, instead of seeing "Sep 6" we see "Sep 0", and instead of "2016-09-06" we see "0000-00-00".
Whether or not this problem is exhibited seems to be determined at app installation time. For both our real-world production app and for a small test app I wrote, the reproduction rate is either 0% or 100% for each installation; bad installations seem to happen 5-10% of the time (though the number of trials is pretty small).
The test app simply creates a LocalDateTime object and then sets some TextViews to display various bits of the date/time. As a sanity check, it also does the same with a Joda object.
private void runOnce() {
    org.threeten.bp.LocalDateTime threetenDate = org.threeten.bp.LocalDateTime.now();
    printThreeten(threetenMonth, "MMMM", threetenDate);
    printThreeten(threetenMonthNumeric, "MM", threetenDate);
    printThreeten(threetenDay, "EEEE", threetenDate);
    printThreeten(threetenDayNumeric, "dd", threetenDate);
    printThreeten(threetenYear, "yyyy", threetenDate);
    printThreeten(threetenTime, "HH:mm:ss", threetenDate);

    org.joda.time.LocalDateTime jodaDate = org.joda.time.LocalDateTime.now();
    printJoda(jodaMonth, "MMMM", jodaDate);
    printJoda(jodaMonthNumeric, "MM", jodaDate);
    printJoda(jodaDay, "EEEE", jodaDate);
    printJoda(jodaDayNumeric, "dd", jodaDate);
    printJoda(jodaYear, "yyyy", jodaDate);
    printJoda(jodaTime, "HH:mm:ss", jodaDate);
}

private void printThreeten(TextView tv, String pattern, TemporalAccessor accessor) {
    try {
        DateTimeFormatter formatter = DateTimeFormatter.ofPattern(pattern);
        String text = formatter.format(accessor);
        tv.setText(text);
    }
    catch (Exception e) {
        e.printStackTrace();
        tv.setText("exception");
    }
}

private void printJoda(TextView tv, String pattern, ReadablePartial partial) {
    try {
        org.joda.time.format.DateTimeFormatter formatter = DateTimeFormat.forPattern(pattern);
        String text = formatter.print(partial);
        tv.setText(text);
    }
    catch (Exception e) {
        e.printStackTrace();
        tv.setText("exception");
    }
}

Good installations show output that looks like
September
09
Tuesday
06
2016
16:46:13

Bad installations show output that looks like
September
exception
Tuesday
exception
0000
exception

Both good and bad installations show the expected output when using the Joda methods.
When an exception is thrown, it is always of the form
09-06 16:46:13.107 21689 21689 W System.err: org.threeten.bp.DateTimeException: Field HourOfDay cannot be printed as the value<PHONE_NUMBER> exceeds the maximum print width of 2
09-06 16:46:13.107 21689 21689 W System.err: 	at org.threeten.bp.format.DateTimeFormatterBuilder$NumberPrinterParser.print(DateTimeFormatterBuilder.java:2361)
09-06 16:46:13.107 21689 21689 W System.err: 	at org.threeten.bp.format.DateTimeFormatterBuilder$CompositePrinterParser.print(DateTimeFormatterBuilder.java:1995)
09-06 16:46:13.107 21689 21689 W System.err: 	at org.threeten.bp.format.DateTimeFormatter.formatTo(DateTimeFormatter.java:1385)
09-06 16:46:13.107 21689 21689 W System.err: 	at org.threeten.bp.format.DateTimeFormatter.format(DateTimeFormatter.java:1359)
09-06 16:46:13.107 21689 21689 W System.err: 	at com.kayak.datetest.MainActivity.printThreeten(MainActivity.java:89)
09-06 16:46:13.107 21689 21689 W System.err: 	at com.kayak.datetest.MainActivity.runOnce(MainActivity.java:75)

The test app shows these exceptions for the fields MonthOfYear, DayOfMonth, and HourOfDay. Only the fields required by the patterns posted above were tested.
It seems that the problem is not the LocalDateTime object itself; it always contains the expected information. Rather, the problem seems to lie in the NumberPrinterParser class, as the bad behavior is only exhibited when trying to print a numeric value. Text values work as expected even on bad installations.
I want to mention also that the YearOfEra field shows different bad behavior in these tests. Printing it never throws an exception, but bad installations print "0000" while good installations print "2016" as expected.
Mirroring @riggaroo's experience, placing a breakpoint on the throw new DateTimeException(...) line causes these problems to stop appearing. As soon as the breakpoint is removed, the problems come back.
I know there aren't any answers in this post, but I hope there might be a few new clues.

@benpm Do you see this happening only on Motorola devices?

@gborland yes, only Motorola. The vast majority of reports are for the MotoG3, which is the phone I used for my tests.

https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatterBuilder.html reads: "This class is a mutable builder intended for use from a single thread."

@jodastephen : was there a code change to fix this before closing the issue?  We still see this in the field w/ the latest version of the lib...(on Android on Moto devices)

No changes. I don't have any way to investigate or reproduce this. So, it is up to those affected to do so and suggest a PR I'm afraid.


Bring up code coverage of controller to 80%

/area test-and-release
/kind doc
Expected Behavior
github.com/knative/build/pkg/controller code cover should be 80+%
Actual Behavior
It's at 50.5%.
Steps to Reproduce the Problem
Pls refer to https://testgrid-dot-knative-tests.appspot.com/knative-build#coverage for up-to-date CC numbers
Additional Info

/assign @shashwathi


Clean-up tables in specification

I have given-up on fixing the tables in the generated PDF. It may be
possible by injecting some Tex directly in the Madoko source file, but
it sounds tedious and hard to maintain. So instead, I have cleaned-up
the tables that could be cleaned-up easily (i.e. these that don't have
long lines of text) using the Madoko online editor, and I have replaced
all other tables with lists. I did prefer the tables (in HTML) but the
lists do look okay IMO and they should be easier to modify & maintain.
Fixes #12

The built spec for this PR is available here: https://s3-us-west-2.amazonaws.com/p4runtime/travis/antonin/clean-up-spec-tables/P4Runtime-Spec.html


Support round of the corners

When generating the icons, we need support to make rounds of the corners.
Example
$ npx pwa-asset-generator logo.svg images --icon-only --icon-rounded 10

Any update? @elegantapp


Fix management of SPI settings

This addresses some problems with creating the SPI settings, which were leading to serial timeout errors when loading the files onto our LED boards.
In a previous version, beginTransaction() was always called with the class's mSettings object. However, mSettings was never properly created as a local object, so a default SPISettings object was created every time beginTransaction was called. In that version, we did not actually have any control over the SPI settings, including the baud rate. In the current version prior to this pull request, the problem was addressed by explicitly creating a new SPISettings object whenever beginTransaction was called. This object was created with the correct baud rate. However, creating the SPISettings object with non-default attributes takes longer than creating the default object, which led to serial timeout errors.
This request fixes that error by correctly creating the SPISettings object mSettings only when the class is first initialized, and whenever the baud rate is changed. This prevents the timeout errors while ensuring we have control over the SPI settings being used.

Thank you Claire - nice sleuthing. Happy new year!


cf install-plugin cflocal-0.17.0-macos FAILED

Tried to install the latest cflocal plugin in my mac and getting the following errors. Any Suggestions.
○ → cf install-plugin cflocal-0.17.0-macos
Searching CF-Community for plugin cflocal-0.17.0-macos...
Plugin cflocal-0.17.0-macos not found on disk or in any registered repo.
Use 'cf repo-plugins' to list plugins available in the repos.
FAILED

Resolved. Executed from wrong directory


[ICE: ] VCX xERC20 is stuck

Describe the bug
xERC20 from vaultcraft is stuck more than 20 hours.
83000$
Link to ticket in Intercom
https://app.intercom.com/a/inbox/qge2vd8z/inbox/admin/7092575/conversation/845
Screenshots

Additional context
Add any other context about the problem here.
@prathmeshkhandelwal1
SLA: 1 workind day

There are a few txns stuck (1 is stuck since 9th may)
https://connextscan.io/tx/0xf456b381fa7901d73d99aff842d1f181f4538d7b81898f468e863c730718c2f7
https://connextscan.io/tx/0x0f5053614a91d7ca24f8c7b8c5bb5167f9a51116772c1088f957d169851c2750
https://connextscan.io/tx/0x6ce09c907cc674b379732a9926b1c6918bcdf533202a7c8a03f6624e6b5069f2
@just-a-node

Integrator sent these txns with bad calldata. In tg chat with this partner trying to clear up the implementation error and figure out how to unstuck the txn.

Need an updated status on it @just-a-node


Addition of a YearView for Schedule

Created a new YearView for the Schedule component. Test site showcasing YearView
Setup as draft request while I ensure that I have pushed everything correctly and to open for discussion.

Did some research and here is more feedback about the UI.
Most other vendors have implemented yearly view as a series of calendars next to each other (that go one below the other on a mobile device). Here is an example from Google calendar:

So basically 12 MonthViews next to each other with a header for the month name. All tested vendors rendered only an indicator (single dot most of the time) when there are events on a date. Clicking the dot behaves like the more button in the current month view - displays a dialog with the appointments for that day.
One of the vendors however had a view that was similar to what you have implemented (they have the google calendar-like year view as well). They called that view YearTimeline though.
For consistency's sake we prefer the YearView to look as in the pasted screenshot for Google Calendar. We believe this is what developers would expect for an early view. However YearTimeline view also seems to have its merits and should not go to waste. I am thinking the following:

Rename it to YearTimeline view.
Do not rotate the month headers to vertical. Keep it horizontal and instead use horizontal scrolling always (the month names should stay fixed as a frozen datagrid column).
Having horizontal scroll always would allow to use longer text for the day names instead of a single letter.
Remove the month headers at the end as they would be unnecessary.
Also remove the text "MONTH" on top of the month headers - seems unnecessary.
Bonus points - implement the Year view. Could probably reuse some of MonthView's implementation.

What do you think?

Hi @akorchev
EDIT - Apologies in advance for reply. I do ramble on if I don't think I'm explaining well enough.
Key for conversation

GV - The Google view in the snapshot that you sent in the last comment
TL  - The Timeline view from one of the other vendors
YP - The proposed Timeline view (was based on regular Year Planner found hanging in many an office)

Absolutely with you on the GV. The ability to stack the separate month views to accommodate different screen sizes is a must. One thing I would do differently to other vendors is I would highlight the whole day if there is an appointment that crosses that day rather than just a dot. I think from a user point of view, it looks quite confusing and doesn't give a clear picture on availability, booked, busy, e.t.c.

My thoughts on TL differ. I had searched whether there were any other vendors that offered a 'Year View' and I did come across what you're referring to myself. I didn't think it was intuitive at all. It didn't give any more than using a Month View and manually 'nexting' through the year. In fact, it may have been less intuitive than 'nexting' through MonthView. I'll try to explain.

With the existing DayView, WeekView and MonthView, they all have context, or a term of reference. Without even looking at the data content, you instinctively know the boundaries and the context of what you are viewing. This was the thinking behind YP. You are viewing a whole year. You barely need to reference any headings, you instinctively know where you are.
With TL, you scroll up, down, left and right, trying keeping one eye on the headers so you can track your position. But even then, there's no context, or term of reference to what you're finally presented with, so the view doesn't really mean anything, or offer anything more than MonthView.
Analogy
It's a bit like a town/city route on Google maps. If you were supplied with a series of zoomed snapshots of a route, you'd be able to find your way to your destination if they were kept in order and with a little work. But if you zoom out to encompass the whole route, you are immediately presented with a view that offers far more. You have terms of reference. Scale. Distance. Could be a couple of landmarks you already know. Although you couldn't guess the exact distance or duration of the route, the landmarks would aid in giving you a very good idea. When presented with the full picture and context, the human brain is extremely adept at such calculations, usually with very little effort from the owner.
My initial thought is that GV shouldn't be too difficult to realise if using the existing MonthView as a starting point. I will have a play around with it over the next couple of days.
As for TL. I think it has the appearance that it's been created by a component designer to plug a gap in a library, rather than offer a real world solution to a problem (just my own personal opinion).
Be good to know your thoughts.
Speak soon and stay well.
Paul

I couldn't understand what you are opposing to. We like the implementation you have created and just want to suggest some changes in the rendering - wider cells in exchange for horizontal scrolling (also would look better on a mobile device).
As for year view vs year agenda - I think it is a matter of preference and both have different use case. One is more of a summary and a very high level overview, the other could serve as a more detailed look over the yearly tasks.

I was worried the changes would have made it a totally different view to what was intended. So, instead, I've created a new view that encompasses all the changes you mentioned above. Now, the existing view is YearPlanner and the changed view is YearTimeline.
Test site

Hi @akorchev
Created "stacked month" year view. Test site
Would like to know your thoughts on layout / usability or any changes you think may improve it.
I need to clean up the code, especially the css. A lot of trial and error using flex display. I will then upload the code as a draft so you can check it out before I set it to ready for review.
Many Regards
Paul

Wow! That's some great progress!
The only thing that bothers me with the Planner view is that it is almost unusable on a mobile device. The cells are misaligned and the headers are on top of each other. Maybe this can be fixed somehow.

Thanks @akorchev
You are probably aware of my reluctance to totally lose the Planner view. It is being used in a real world application (desktop only) where scrollable wouldn't have been an option and I do think it has it's place.
I do see how you need controls to be able to be used on any device without the developer having to program for the different media. Unfortunately, having all the data on the screen on a small device is just not practical. By it's design definition, it really is just for a desktop or large media device.
So how does this sound? We combine the Planner and Timeline views into one. We add a property called "FullScreen", which will affect the control like this -
if FullScreen == true
if ScreenSize (or container size) > PredefinedMinimum
Show Planner
else
Show Timeline
else
Show Timeline
This will allow a developer to show a full screen planner view if they wish, but will fall back gracefully to Timeline if displayed on small form factor device.
Many Regards
Paul

Hi @akorchev
I think it's just about there Test Site
I've used a few @media queries so that on small devices the cells are fixed and scroll bars are used, whilst on larger devices, the cells will expand or shrink to fit the container.
Regards
Paul

Hey @paulo-rico,
This looks a lot better indeed! Great work. I found one small issue that I am sure is easy to fix. There is some white space between the rows after making the window smaller:

Hi @akorchev
I think it's just about there (again) Test Site
Limited height of content so it cannot stretch past a certain threshold.
Tried many different views and zooms in Dev Tools and looks to be o.k.
Many Regards
Paul

It definitely looks better. The only thing that bothers me is that there just isn't enough room for everything:

Here it isn't clear which cell this "+3 more" applies to. Also if this has to appear in two adjacent cells overlaps will occur. Maybe the text should just be "..." so it can fit in the cell always.

MoreText changed to "+{0}" Test Site
Many Regards
Paul

The text is still outside of the event though. I think the slot needs a bit more height or padding:

Hi @akorchev
A mistake in my media queries. I'm going to clean up the CSS as best I can from this end, but definitely need someone with experience in this field to sort it out.
My initial clone is probably battered after the alterations I've been making in the last few weeks. There are a couple of changes in one existing razor.cs file and then the changes to the _scheduler.scss file. The rest are all new files.
I think I may have to park this clone, get a fresh clone, and then paste my changes in.
What's the best way of doing this?
Thanks for your patience
Paul

Hi @akorchev
Test Site
I removed the original commit and worked from a fresh clone. Managed to sort a great deal on the css. I'd created that many incremental changes through our back and forths, that I think I lost my way. Using @media query, I could just use same css as MonthView for large full-screen display, and then use a seperate css for small screens that scrolls.
I think the css still needs to be tidied up by a css guru.
Regards
Paul

Thank you so much @paulo-rico, we will review the CSS later this week and merge your PR.


Arbi-Bom: Upgrade the devstack to support  4.4 in edx-platform

Description
As a devstack user I can test my service on Mongo 4.4 so that I can be ready for upgrading other environments.
AC:

Modify this script to update to 4.4 in devstack and ask SRE to review and test.
Ensure we can test import and export a course using devstack.
ENSURE we post a message in dev, email to eng-all (template here) and if needed a post to https://discuss.openedx.org/.

Implementation Details:

SRE have context on this work.
Same ticket needs in cs-forums.

SRE is finding a solution for backup for Mongo 4.4, so this will be blocked until a new solution is found.


Fix remaining chunk processing in optimized.py

Fixes https://github.com/benhoyt/countwords/issues/28

Thanks!


bug second mi-robot has no map

Hello Guys,
i have two instances for the two robots with original firmware.
The first one create a link and a picture for the map.
But the second one leave the datapoints empty.
Controlling the second robot works and other datapoint where filled.
2,1,1 is installed.

Same here!
Software 2.1.1
S4: -> vacuum.0 = working map (/mihome-vacuum.admin/actualMap_0.png)
S6 pure: -> vacuum.1 = non working map ()

Same here!
Software 2.1.1
S4: -> vacuum.0 = working map (/mihome-vacuum.admin/actualMap_0.png)
S6 pure: -> vacuum.1 = non working map ()

Hello,
Same for me...
Adapter v 2.2.5
2x S5

fixed in 3.1.1


Face detection and Text Analysis were added

Hello guys,
I added the Face detection and Text analysis to the application. You can see the screenshots in the below.
My thoughts about the Cognitive Service is really good. It is really easy to apply on our projects. Documentations explains almost everything. I have not applied the Custom Vision on my project before,  with this challenge, I had a chance to look the codes about applying custom vision.
I did not have any problem while applying the cognitive services, it was fun to apply cognitive services on app.

Thanks for completing the challenge! Fill out this form so we can send you the swag! https://aka.ms/cog-svcs-swag


Improve launch performance by using hyperfine

Write an decision doc to show how to use hyperfine to profile startup to the v-sekai.

Superseded by https://github.com/V-Sekai/V-Sekai.github.io/issues/218.


explicitly endorse CC-BY

I know Andreas that you like CC-BY best, however, I think we should endorse CC0 instead because I think good academic practice should warrants a citation/mention of the authors and not licence law.
I'll rewrite it for CC-BY away, but if you are persuaded let me know.

Good point. I agree that for academic purposes, the rules of good scientific practice to which scientists are committed, should guarantee proper citation. However, open materials/manuscripts/etc are released to the entire world including the corporate world, which need to be reminded (if necessary by a legal disclaimer) that they need to acknowledge the source they draw materials from.

OK, you have convinced me that such a recommendation would do more good than harm.


Solution not printing due to nans in vector variable

It seems that the solution isn't printing is because of the nan's. Notice that these nan's appear where the vector element value has been substituted, and it doesn't matter what the substituted value is.
So this issue has two parts:

the vector variables have nan's where they should have the value of eps.
the solution doesn't print when some of it's values contain nan's

Code:
from gpkit.shortcuts import *
from numpy import ones

N = 6
L = 5.
dx = L/(N-1)

EI = Var("EI",10)

P = 100
p = Vec(N, "p", descr="Distributed load")
p = p.sub(p, P*ones(N))

V  = Vec(N, "V", descr="Internal shear")
M  = Vec(N, "M", descr="Internal moment")
th = Vec(N, "th", descr="Slope")
w  = Vec(N, "w", descr="Displacement")

eps = 1E-6 #something small

V[-1] = V[-1].sub(V[-1], eps)
M[-1] = M[-1].sub(M[-1], eps)
th[0] = th[0].sub(th[0], eps)
w[0]  = w[0].sub(w[0], eps)

objective = w[-1]

constraints = [EI*V.left[1:N]     >= EI*V[1:N]    + 0.5*dx*p.left[1:N]     + 0.5*dx*p[1:N],
               EI*M.left[1:N]     >= EI*M[1:N]    + 0.5*dx*V.left[1:N]     + 0.5*dx*V[1:N],
               EI*th.right[0:N-1] >= EI*th[0:N-1] + 0.5*dx*M.right[0:N-1]  + 0.5*dx*M[0:N-1],
               EI*w.right[0:N-1]  >= EI*w[0:N-1]  + 0.5*dx*th.right[0:N-1] + 0.5*dx*th[0:N-1]]

gp = GP(objective, constraints)

sol = gp.solve()

print sol.table()

Output:
In [17]: %run beam.py
Using solver 'mosek'
Solving for 20 variables.
Solving took 0.0283 seconds.

Cost
 0.781

Variables
---------
EI : 10

Constant sensitivities
----------------------
EI : -4

If I inspect sol, the "second" solution seems to be:
'variables': {EI: array(10),
  w: array([        nan,  0.05125112,  0.18500127,  0.36625144,  0.57000162,
          0.78125184]),
  M: array([ 12.5000031 ,   8.00000206,   4.50000166,   2.00000138,
           0.50000117,          nan]),
  V: array([ 50.00000892,  40.00000314,  30.00000193,  20.00000143,
          10.00000115,          nan]),
  th: array([        nan,  1.02500128,  1.65000149,  1.97500168,  2.10000187,
          2.12500215])}

Smaller minimum working example:
from gpkit import Variable, VectorVariable, GP
x = VectorVariable(5, 'x')
x[0] = x[0].sub(x[0], 1e-3)
prob = GP(sum(x), [x[1:] >= 1])
sol = prob.solve()
print sol.table()
sol["variables"]

Output:

Using solver 'cvxopt'
Solving for 4 variables.
Solving took 0.00616 seconds.

Cost
----
 4.001

Free variables
--------------

Swept variables
---------------

Constants
---------

Constant and swept variable sensitivities
-----------------------------------------

Will work on this today

Not that you need advice, but I'd probably start by inspecting x[0] after the sub, as well as sum(x).

The problem arises due to _parse_result and I am working on a fix. But I have a question about the end goal here.
Consider the following GP. It is equivalent to the MWE:
from gpkit import Variable, GP

x0 = Variable('x0')
x1 = Variable('x1')
x2 = Variable('x2')
x3 = Variable('x3')
x4 = Variable('x4')

x0 = x0.sub(1e-3)

gp = GP(x0 +  x1 + x2 + x3 + x4, [x1 >= 1., x2 >= 1., x3 >= 1., x4 >= 1.])
sol = gp.solve()
print sol.table()

When you solve this, it will print a solution that looks like:
Free variables
--------------
x1 : 1
x2 : 1
x3 : 1
x4 : 1

My design question is whether the VectorVariable solution to the MWE should print as follows:
Free variables
--------------
x : [ 0.001     1         1         1        ... ]

This is ever so slightly different and inaccurate because the first element is not free, but also makes the most sense from the "Vector Variable" perspective.

What if...there was a plain "mixed variables" category for such edge cases?
Ugh, that also seems gross.

Yeah I agree that's kind of nasty. What about an asterisk to denote that a particular element is fixed?

So, just display nans as asteriks?

Err.. I guess that could work? But I meant something like:
Free variables
--------------
x : [ 0.001*     1         1         1        ... ]

*values denoted with an asterisk are fixed
Or something.

But how do we tell between a partly-substituted free variable and a
partly-free constant? :p

Valid.
By the way, it's unclear to me: does _parse_result() see the substituted element at all? If so, I can't seem to find where.

The design issue is a fair point, and I don't have strong feelings about the right solution. A simple solution, which I'm fine with, is printing any VectorVariable as a free variable if it has any free variables inside it. But a more complicated scheme like the asterisks is also fine with me.
I think you guys are a step ahead of me -- is it clear what's introducing the nans?

Yes, sorry, it's the following excerpt of code within _parse_result():
try:
    variables[veckey][idx] = variables[var]
    sensitivities["variables"][veckey][var.descr["idx"]] = \
                        sensitivities["variables"][var]
except KeyError:
    variables[veckey] = np.empty(var.descr["shape"]) + np.nan
    sensitivities["variables"][veckey] = (
                        np.empty(var.descr["shape"]) + np.nan)

This code is in a loop that essentially takes
variables = {x1: 1.0, x2: 1.0,... x4: 1.0}

and transforms it into
variables = {x = [1.0, 1.0, 1.0, 1.0]}

That particular segment of code is creating the nan-initialized list for x, if it doesn't exist already.
The problem is that it knows that x should have 5 elements, but only knows what 4 of them are (see the original variables, so what it is actually producing, as we have seen is
variables = {x = [nan, 1.0, 1.0, 1.0, 1.0]}

Please can someone explain the following segment of code to me:
         # var does not exist yet
         for var, S in sensitivities["variables"].items():
             if var not in self.substitutions and abs(S) > 1e-4:
                 raise RuntimeWarning("free variable too sensitive:"
                                     " S_{%s} = %0.2e" % (var, S))

If I add a couple print statements afterwards, I get the following output:
         print sensitivities["variables"].items()
         # [(x_(4,), 0.0), (x_(1,), 0.0), (x_(2,), 0.0), (x_(3,), 0.0)]
         print var.descr["shape"]
         # (5,)

How is var.descr being defined?

Is an acceptable solution just to have the following output:
Free variables
--------------
x_(1,) : 1
x_(2,) : 1
x_(3,) : 1
x_(4,) : 1

(ignores substituted element entirely)

I have identified why the solution doesn't print when it contains NaNs.
This is due to minval in small_scripts.results_table(). Specifically, the following line causes the issue:
Removing the if statement resolves this issue. However, minval clearly serves a purpose, so I propose adding an or statement that allows nans to pass through. This would fix half of this issue. How would you feel about this @whoburg, @bqpd ?

I have identified why the solution doesn't print when it contains NaNs.
This is due to minval in small_scripts.results_table(). Specifically, the following line causes the issue:
decorated = [(bool(v.shape) if isinstance(v, Iterable) else False,
                 (varfmt % k),
                 i, k, v) for i, (k, v) in enumerate(data.items())
                 if np.max(abs(v)) >= minval]

Removing the if statement resolves this issue.
decorated = [(bool(v.shape) if isinstance(v, Iterable) else False,
                 (varfmt % k),
                 i, k, v) for i, (k, v) in enumerate(data.items())]

However, minval clearly serves a purpose, so I propose modifying the conditional so that it specifically allows nans to pass through. This would fix half of this issue. How would you feel about this @whoburg, @bqpd ?

Also, I know it's not my place to criticize someone else's code, but it physically hurts me to read that line.
It's like everything bad about python in 4 nasty, ultra-compressed lines.

Hahaha. I can laugh because that's one of @whoburg's fixes to the terrible mess that came before.

You could do this with np.max(abs(v)) >= minval or isnan(np.max(abs(v))), I think...

yup done, I'm just creating a pull request now

Interesting discover about the MWE.
Code 1 (unmodified MWE):
from gpkit import VectorVariable, GP
x = VectorVariable(5, 'x')
x[0] = x[0].sub(x[0], 1e-3)
prob = GP(sum(x), [x[1:] >= 1])
sol = prob.solve()
print sol.table()

Output 1:
Cost
----
 4.001

Free variables
--------------
x : [ nan       1         1         1        ... ]

Swept variables
---------------

Constants
---------

Constant and swept variable sensitivities
-----------------------------------------
x : [ nan       0         0         0        ... ]

Code 2:
from gpkit import VectorVariable, GP
x = VectorVariable(5, 'x')
prob = GP(sum(x), [x[1:] >= 1], {x[0]: 1e-3})
sol = prob.solve()
print sol.table()

Output 2:
Cost
----
 4.001

Free variables
--------------
x : [ 0.001     1         1         1        ... ]

Swept variables
---------------

Constants
---------

Constant and swept variable sensitivities
-----------------------------------------

My conclusion is therefore that the problem isn't in geometric_program as I thought all this time, but rather in the Variable.sub() method.

This issue is also true for a non-vector-variable case:
x0 = Variable('x0')
x1 = Variable('x1')
x2 = Variable('x2')
x3 = Variable('x3')
x4 = Variable('x4')

x0 = x0.sub(1e-3)

gp = GP(x0 +  x1 + x2 + x3 + x4, [x1 >= 1., x2 >= 1., x3 >= 1., x4 >= 1.])#, {x0: 1e-3})
sol = gp.solve()
print sol.table()

Cost
----
 4.001

Free variables
--------------
x1 : 1
x2 : 1
x3 : 1
x4 : 1

Swept variables
---------------

Constants
---------

Constant and swept variable sensitivities
-----------------------------------------

vs.
x0 = Variable('x0')
x1 = Variable('x1')
x2 = Variable('x2')
x3 = Variable('x3')
x4 = Variable('x4')

#x0 = x0.sub(1e-3)

gp = GP(x0 +  x1 + x2 + x3 + x4, [x1 >= 1., x2 >= 1., x3 >= 1., x4 >= 1.], {x0: 1e-3})
sol = gp.solve()
print sol.table()

Cost
----
 4.001

Free variables
--------------
x1 : 1
x2 : 1
x3 : 1
x4 : 1

Swept variables
---------------

Constants
---------
x0 : 0.001

Constant and swept variable sensitivities
-----------------------------------------

In fact the following code is also different:
x0 = Variable('x0', value=1e-3)
x1 = Variable('x1')
x2 = Variable('x2')
x3 = Variable('x3')
x4 = Variable('x4')

gp = GP(x0 +  x1 + x2 + x3 + x4, [x1 >= 1., x2 >= 1., x3 >= 1., x4 >= 1.])
sol = gp.solve()
print sol.table()

Cost
----
 4.001

Free variables
--------------
x1 : 1
x2 : 1
x3 : 1
x4 : 1

Swept variables
---------------

Constants
---------
x0 : 0.001

Constant and swept variable sensitivities
-----------------------------------------

This seems to contradict the doctstring in Variable.sub() that claims
assert x.sub(3) == Variable('x', value=3)

That docstring is wrong, sorry. I think it may have come from an earlier design discussion @whoburg and I had.

I think I'm close to fixing the remainder of this issue.
Essentially, because the Variable.sub() method is returning a Monomial instead of a Variable, the objective function is then passed to GP() as 0.001 + x1 + ... instead of x0 + x1 + ..., which eventually leads to the parse_results() missing x0 entirely.
Unfortunately, I don't think I know of an elegant way of fixing Variable.sub. I want to do something really simple like self.desc["value"] = arg, but this breaks down for vector variables.
Any and all suggestions are appreciated! @whoburg @bqpd
Design question: do we even need 3 separate ways of performing substitutions?

This is a great example of a really excellent discussion / workflow! Thanks @pgkirsch for helping to nail down the issue(s).
In my opinion the important issue is not how many ways of performing substitution we have, but rather that they all have the same behavior/syntax/api. This seems like an example that breaks that, since passing a substitution dict into the GP worked, but substituting the variable beforehand didn't.
This is definitely related to another ticket where @bqpd and I were discussing what x.sub(3) should do -- need to track that down.

The related issue is #267.
@bqpd, any thoughts? Is sub perhaps not what users should be using to say that a Variable (and especially an element of a VectorVariable) has a value? Or is this a bug / design flaw?

I think Variable.sub should fall back on Signomial.sub as part of the Signomial inheritance contract. If something's a Signomial, subbing into it replaces varkeys instead of setting their values.

I'm confused because isn't that already what happens?

Yeah, sorry for the confusion. I was answering @whoburg's "Is sub perhaps not what users should be using to say that a Variable (and especially an element of a VectorVariable) has a value?" with the main reason Yes.

Ok, I'm fine with Variable.sub falling back on Signomial.sub. And it seems indeed that VectorVariable is handling sub in a reasonable way:
In [1]: from gpkit import VectorVariable

In [2]: x = VectorVariable(3, 'x')

In [3]: x
Out[3]: gpkit.PosyArray([x_(0,), x_(1,), x_(2,)])

In [4]: x[0] = x[0].sub(4)

In [5]: x
Out[5]: gpkit.PosyArray([4, x_(1,), x_(2,)])

In [6]: x.sum()
Out[6]: gpkit.Posynomial(4 + x_(1,) + x_(2,))

It seems to me the place things start to go wrong is the nans in the GPSolutionArray. From a user's perspective, it seems the natural design would be for substituted constants in a VectorVariable to also appear in the solution array. Does that break or design philosophy or inheritance or substitution in some fundamental way?
A weird edge case is a fully-substituted VectorVariable, which of course no longer contains VarKeys...

Ok, thanks to the fixes so far (1238df57a7059f390a9c332bb1e10eb75d24f4a1), the solution to the original problem now prints, with nans.
I'm changing the title and moving this to 0.3.0. Next steps are

[ ] Decide whether a GPSolutionArray should ever contain nans.
[ ] Recommend best practices for setting elements of a VectorVariable to a constant

I think that a GPSolutionArray should contain nans to indicate missing values: if you try to e.g. do vector multiplication with an incomplete vector, the nans will propagate through nicely.
Best practices is definitely a substitution dictionary, as below:
from gpkit.shortcuts import *
from numpy import ones

N = 6
L = 5.
dx = L/(N-1)

EI = Var("EI",10)

P = 100
p = Vec(N, "p", descr="Distributed load")
p = p.sub(p, P*ones(N))

V  = Vec(N, "V", label="Internal shear")
M  = Vec(N, "M", label="Internal moment")
th = Vec(N, "th", label="Slope")
w  = Vec(N, "w", label="Displacement")

eps = 1E-6 #something small

substitutions = {var: eps for var in [V[-1], M[-1], th[0], w[0]]}

objective = w[-1]

constraints = [EI*V.left[1:N]     >= EI*V[1:N]    + 0.5*dx*p.left[1:N]     + 0.5*dx*p[1:N],
               EI*M.left[1:N]     >= EI*M[1:N]    + 0.5*dx*V.left[1:N]     + 0.5*dx*V[1:N],
               EI*th.right[0:N-1] >= EI*th[0:N-1] + 0.5*dx*M.right[0:N-1]  + 0.5*dx*M[0:N-1],
               EI*w.right[0:N-1]  >= EI*w[0:N-1]  + 0.5*dx*th.right[0:N-1] + 0.5*dx*th[0:N-1]]

m = Model(objective, constraints, substitutions)

sol = m.solve()

print sol.table()


Unable to use Residential proxies.

Am trying to use residential proxies with seleniumbase. They come in the form of hostname:port:username:password
e.g. proxy.iproyal.com:8082:usernameString:0qMspCfWtAFErhA9_country-us_session-SyrZinpW_lifetime-1h.
Please guide me on how to use this. I am using the SB format.

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

You can add proxy="USER:PASS@SERVER:PORT" or proxy="SERVER:PORT". Eg:
from seleniumbase import SB

with SB(proxy="USER:PASS@SERVER:PORT") as sb:
    #   ...

Thanks so much @mdmintz, it works and I appreciate you!


Minor: Add multi ordering test for array agg order

Which issue does this PR close?

Rationale for this change
I found the reason why we need level 3 vec in convert_array_agg_to_orderings https://github.com/apache/arrow-datafusion/pull/7629/files#r1357637090. For multi ordering cases, we have len > 1.
Add multi ordering test to improve the test coverage, also simply the code.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Thanks again


Adapt Docker development env to namespace changes

This PR simply adapts the paths used by the development Docker to the new structure introduced in https://github.com/3scale/apicast/pull/486

Right. I'll add that in a future PR.


Feature/device code

Following the email discussion, here is another sample for the device code.
TODOs:

registration instructions & screenshots
day number

@baywet Do you want to put the Device Code changes and documentation into a new subfolder for Day 22, then we can separate out the Planner code into a Day 23 (numbers can be modified later as needed)?  Want to make sure not losing the Planner work that was merged already.


Migration Script and Other enhancements

Adds a migration script with many features
Adds option to Disable the XMPP server completely.
Uses AdminSDK as the default method of sending notifications since XMPP does not provide delivery receipts anymore so no evident advantage in using it. We may deprecate XMPP in the future or revert to it if we feel it was more stable.
Adds an endpoint to delete user
Use HIGH priority for sending notification messages so they are not affected by doze mode in android.
Adds logging to different files for INFO and ERROR level.
Misc Fixes

Merging this now as needed for deployment in alpha


[Feature] Generate App Key with instructions on first load.

Is your feature request related to a problem? Please describe.
With the introduction of an APP_KEY requirement in the configuration, users are now directed to generate a key when setting up.
This can be conveniently done at the address provided in the documentation. But this requires users to go to an external resource to set up the container.
Describe the solution you'd like
There exists the capability to generate a key from the containers console.
I suggest to check on load if the key is not provided, and if blank, generate a key for the user and print to screen with instructions to place it in the relevant ENV variable and reboot.
This way users can setup the container without having to go to an external resource.

So this used to be the case where we'd generate a key on first boot but because people either store these keys as environment values in their docker-compose.yml files or physically in a .env it was more complex to cover the use cases.
I don't have any plans to revert back to that patten but I will keep it in mind going forward if an opportunity arises to improve the initial install. Thanks for the feedback!

So I’m not talking about changing that behaviour.
Just adding a print to the log that generates a key if null, with a note telling the user, “APP_KEY missing, one has been generated for you below, please place into docker ENV variable and reboot container”.
That way users that don’t populate a key will be provided instructions and a key to do so.
Users that do, well nothing will have changed.


Scheduling the update process?

Hi there,
I would like to know if it is possible to schedule the update run.
I have been caught by a CPU spike at the wrong time of my day and found it was the ruby process of this workflow that I believe is doing an update to the WF or the cache or both.
Is there anyway I can configure this to not happen randomly during my day and instead have it done overnight?
My machine is never off...well almost never off.
Thanks!

It’s not random, it happens at 8 p.m. To change it, edit these lines in ~/Library/LaunchAgents/com.alfredapp.launchd.googledrivecache.plist

Thanks for the fast response.
Do I need to restart anything, reload anything after changing the file or
will it automatically get picked up?
On Thu, Jan 20, 2022 at 8:24 PM Vítor Galvão @.***>
wrote:

Closed #24 https://github.com/alfredapp/google-drive-workflow/issues/24.
—
Reply to this email directly, view it on GitHub
https://github.com/alfredapp/google-drive-workflow/issues/24#event-5924467569,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQW7CTZK6OFG6QUSJY5V3DUW75I5ANCNFSM5MMS2FVA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***
com>

--
Best regards,
-FF

Either reboot or open a terminal and:
osascript -e 'tell application id "com.runningwithcrayons.Alfred" to run trigger "googledrivelaunchd" in workflow "com.alfredapp.googledrive" with argument "uninstall"'
osascript -e 'tell application id "com.runningwithcrayons.Alfred" to run trigger "googledrivelaunchd" in workflow "com.alfredapp.googledrive" with argument "install"'

You need to reboot your machine.


docs: [IAC-2722]: Add STO license requirement to WIZ highlights

Description

Add note that Wiz scans require an STO license.
JIRA

PRs must meet these requirements to be merged:

[x] Successful preview build.
[x] Code owner review.
[x] No merge conflicts.
[x] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6723a8db20bb2f3a1c2078b9--harness-developer.netlify.app


CSV import doesn't support escaped commas

When trying to import a CSV file that has data with escaped commas, you get extra columns.
Example CSV
name,description
Test,"Hi, this doesn't work"

Expected
The table is built with two columns like this:
name | description
-----+--------------------
Test | Hi, this doesn't work

Actual
We get an extra column like this:
name | description |
-----+-------------|------------------
Test | Hi          | this doesn't work

Hi, this has now been updated in version 1.1.3 where the commas are parsed correctly and quotes are also retained.
Let me know if this is sufficient. Thanks!


Implement np.kron.

It's true, that hadn't occurred to me. Einsum isn't a natural part of my numpy vocabulary. That said, this works now, so I'm going to merge it and move on :-)


Change groupBy logic for aggregatorServer in spring turbine amqp

As per default implementation, we are getting aggreagted dashboard with all instances present in RabbitMq channels. How can we update this logic to aggregate the instances ? As we want to make one cluster per application so can we have one cluster per channel of rabbit mq?
I am referring this code -
                             Observable<Map<String, Object>> publishedStreams = StreamAggregator
			.aggregateGroupedStreams(hystrixSubject().groupBy(
					data -> InstanceKey.create((String) data.get("instanceId"))))
			.doOnUnsubscribe(() -> log.info("Unsubscribing aggregation."))
			.doOnSubscribe(() -> log.info("Starting aggregation")).flatMap(o -> o)
			.publish().refCount();

Duplicate of #166

I am not able find any concrete example for this purpose with issue #166 . As we need to filter the data in the turbine server anyway (not the messaging layer) so here we have to configure the metrics generation as well in -
        @Scheduled(fixedRateString = "${hystrix.stream.queue.gatherRate:500}")
                public void gatherMetrics() {
                        JsonGenerator json = this.jsonFactory.createGenerator(jsonString);
                    }

where we can add some parameter which can be used in our turbine server to filter this.
Do we have such concrete example as I am not getting that thing configured?/

Turbine already has data, the cluster, to group by. I'm not keen on adding a new way to group by.

Can we have clusters while using spring cloud turbine amqp or stream as well?
Like here we are getting all service instance all together on a single dashboard. How can we configure cluster in rabbit mq implementation ?

Like I said, that feature doesn't exist yet and is the purpose for #166.


HIP 70: Returns cooldown reduction

Based on conversations with various community members, we've come to a compromise on cooldown reduction. With a 3 month lock up and an expected earliest implementation of a chain migration at the end of the year, this should allow mining pools to make a short term decision on unstaking and returning funds to constituents.
We also remove the need for vesting HNT as all staked HNT now becomes veHNT after the chain migration.

#471


[BUG] Raspberry Pi 5 error setting up environment : Unsupported system page size when setting Python environment

Reproduction steps
This is unique to Raspberry Pi 5.
This is both a problem report and a workaround, but it took quite a while to find a workaround, so if it is possible to have this fixed so the workaround isn't needed, that may help others.
I'll start with a workaround so it doesn't get lost in the text below.
To avoid a jemalloc error on RPi 5 when setting up the Matter build environment, alter the file  /boot/firmware/config.txt, to add the lines:
#  Start kernel with 4K page size instead of 16k
kernel=kernel8.img

Now here's how the underlying issue is encountered:
On a Raspberry Pi5, with latest Raspian OS / all updates installed, when creating the build environment per these instructions: https://github.com/project-chip/connectedhomeip/blob/master/docs/guides/BUILDING.md#prerequisites , the step:
source scripts/bootstrap.sh

results in the error:
Downloading and installing packages into local source directory:

  Setting up CIPD package manager...done (4.9s)
  Setting up Project actions........skipped (0.1s)
  Setting up Python environment.....[-]
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
['gn', 'gen', '/home/jvm33/connectedhomeip/.environment/gn_out', '--args=chip_crypto="boringssl" dir_pigweed="/home/jvm33/connectedhomeip/third_party/pigweed/repo"']

Traceback (most recent call last):
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 402, in install_packages
    subprocess.check_call(
  File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gn', 'gen', '/home/jvm33/connectedhomeip/.environment/gn_out', '--args=chip_crypto="boringssl" dir_pigweed="/home/jvm33/connectedhomeip/third_party/pigweed/repo"']' died with <Signals.SIGABRT: 6>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 995, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 987, in main
    return EnvSetup(**vars(parse())).setup()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 577, in setup
    result = step(spin)
             ^^^^^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 766, in virtualenv
    if not virtualenv_setup.install(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 443, in install
    install_packages(gn_target)
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 410, in install_packages
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['gn', 'gen', '/home/jvm33/connectedhomeip/.environment/gn_out', '--args=chip_crypto="boringssl" dir_pigweed="/home/jvm33/connectedhomeip/third_party/pigweed/repo"']' died with <Signals.SIGABRT: 6>.
Installing pip requirements for all...
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Bug prevalence
When activating the bootstrap script on RPi 5.
GitHub hash of the SDK that was being used
latest Matter SDK as of Jan. 12, 2024
Platform
raspi
Platform Version(s)
RPi 5
Anything else?
No response

Has annyone fixed this?

Boostrap should try to create a separate virtualenv (in .virtualenv if PW_ENVIRONMENT_ROOT  is not set) and use that.
The error you have seems to say "you should not install system python packages and instead use the system package manager like dpkg or similar" which is true, however bootstrap should have created a virtualenv for that. Guessing that maybe virtualenv package is not available?
Will need to debug this a bit, however for starters does this do anything for you?
virtualenv ~/pyenv

This should generate a virtual environment in ~/pyenv. If this does not work, probably a python virtualenv package is missing and maybe that is the error.
if it does work, you can try to:
source ~/pyenv/bin/activate.sh
source boostrap.sh

and then pip installs should happen in pyenv instead of systemenv. ... however this is NOT a correct workaround: bootstrap should create its own virtualenv, so we have to figure out why it does not.

Good morning
I am encountering the very same error on Raspberry Pi 5..
However @jvmahon , I am unable to find the file "/boot/firmware/config.txt"
Could you give a little bit more detail on where to find it? Is it mean to be from the root of connectedhomeip/boot/firmware/config.txt ?
Thanks a lot..

However @jvmahon , I am unable to find the file "/boot/firmware/config.txt"

I think /boot/firmware/config.txt might be  link to /boot/config.txt
Can you find a file  /boot/config.txt?
Try editing that instead.
Its been a long time since I originally posted this and am no longer using RPi 5 for a  development environment, so I can't even check what I did right now (sorry). You should be able to google for jemalloc error kernel8.img and find others who have solve this (that's how I originally found a solution).


DEP: remove data module, fixes #93

Codecov Report

Merging #110 (2b12ff1) into main (2878a28) will increase coverage by 1.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #110      +/-   ##
==========================================
+ Coverage   75.96%   77.09%   +1.13%
==========================================
  Files          43       41       -2
  Lines        2159     1943     -216
==========================================
- Hits         1640     1498     -142
+ Misses        519      445      -74

Impacted Files
Coverage Δ

src/crowsetta/__init__.py
100.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4e69e6...2b12ff1. Read the comment docs.


PRs to implement Solar APIs?

Thanks for this project!  I don't have a Tesla vehicle, but I do have Tesla solar panels.  Wondering if you'd be open to some PRs to add these missing apis:
https://www.teslaapi.io/energy-sites/state-and-settings
I've got some working locally already -- just wondering whether to fork/augment this project or contribute directly.  Thanks again!

Sure! I am open to PR's for solar. It is something that I've wanted to do but I don't have the solar panels :)

🎉 Awesome!  PRs coming soon...

Please be sure to add/update docs. And if you are familiar with apiary you can update the API definition and add test cases.

Thanks @mseminatore - Any idea why I'd be getting a 429 "Too Many Requests" error on some of the tests?  Seems to happen on 19 tests but the rest pass.
$ node -v
v12.9.1
$ npm -v
6.10.2
$ export TESLAJS_SERVER=http://private-111a5-teslajs.apiary-mock.com
$ export TESLAJS_STREAMING=http://private-111a5-teslajs.apiary-mock.com
$ export TESLAJS_LOG=1
$ npm ci
$ npm test
  167 passing (20s)
  19 failing

  1) TeslaJS
       #mediaVolumeDownAsync()
         should succeed:
     Error: the string "Error response: 429" was thrown, throw an Error :)
      at tryCallOne (node_modules/promise/lib/core.js:37:12)
      at /Users/dcadwallader/dev/forks/TeslaJS/node_modules/promise/lib/core.js:123:15
      at flush (node_modules/asap/raw.js:50:29)
      at processTicksAndRejections (internal/process/task_queues.js:75:11)

[snip]

To protect their servers from DOS attacks many sites will throttle request rates. This suggests that the tests are calling apiary too quickly. I am not sure if there is a way to slow down requests in Mocha

@geekdave Any updates on progress? Still getting 429 issues?

@geekdave is this something that you are still working or shall I close this out?

@mseminatore finally got around to wrapping this up... let me know if any changes are needed!


Payments - Error message displayed when selecting the any bank account after back online

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Issue found when executing PR https://github.com/Expensify/App/pull/17136
Action Performed:

On any devices, login to an Expensify account
Go to Setting > Payments
Click Add payment method > Bank account
Proceed with Plaid identification until selector page is displayed
Go offline
Go online
Select any bank account and click continue
Verify that User can proceed with submitting chosen account.

Expected Result:
User can proceed with submitting chosen account.
Actual Result:
Error message displayed when selecting the any bank account after back online
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: n/a
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/231864661-15bb2ac7-b976-426e-8b15-9e65fc503a63.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

Triggered auto assignment to @arielgreen (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

I tried a few times the same flow as in the video and can't reproduce the error. I'm thinking it could have been a once-off issue, could you try again please? If it keeps happening, please also provide the email address of the test account you're in so that we can check logs in our backend please 🙌

@arielgreen Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

@kbecciv Can you try this one again please?

This was not repro on staging again https://expensify.slack.com/archives/C9YU7BX5M/p1681768782720529?thread_ts=1681765190.090689&cid=C9YU7BX5M
Closing for now.


Fix for font issues on linux machines

It's been noticed a few times, and being on linux (Ubuntu, particularly) seems to cause the same issue with the font bundles defaulting to the sans-serif.
This patch changes locally hosted lato ttfs for google font hosted woff2, with extended latin subsets. (which may also be the issue, due to localisations).
Closes #6661

@hallaathrad Thanks but we can't use Google as a hosting platform as it is blocked in China.

right... how about I fix this so we have the woff2 locally instead?

I've amended the PR to host the woff2 files in the server (if this works, I'll remove the ttf)...
improved the css for (non-extended) latin characters.

woff2 is not supported in IE/EDGE, or Safari

Trying a different approach: browser specific font CSS.


Interface for plugins

Interface with standard methods must be defined

Plugins have to use annotations to define there standard methods.


Add template config for external users

Closes #111.
Here I am adding in a template config file that can be used as a guide for external users to set up their own configuration settings. I am leaving this file in the root directory, but I had also thought about moving it to the config folder. If anyone has opinions on that, I'm happy to move it.
The first section includes the parameters that they would need to alter, specifically the run_metafile and the outdir. I did not include other parameters as those are not something that everyone would need to change.
I did include the parameter for the spatial containers with a note that the spaceranger container would only need to be updated in the event that the user is using the spatial transcriptomics workflow. Are we okay with including this here?
The second section contains the profiles section which will need to be updated depending on how they plan to run the workflow. Here, we are showing an example of running with slurm. Only the executor and queue are included as the allocation of resources (like mem and cpu) are loaded through process_base.config. Is there anything here that you think I am missing that we should be sure to include?
The last section is to enable singularity in the event that the environment they would like to run the workflow on uses singularity rather than docker. There are instructions in the comments to remove that section if they are going to be using docker.
I believe I covered all of the necessary items that would need to be changed for users trying to run the workflow externally in a different setting, but if there are any obvious things that I am missing please let me know. Also please take a look at the instructions or comments in the file to make sure that they are clear.

Thanks for these really helpful suggestions @jashapiro. I changes the file name per your suggestion. I can't really think of anything that's better and also still short. I also added in an example at the top of using this config file with the workflow and made some minor wording adjustments throughout, adding a description of using either run_ids, library_ids, or sample_ids for params.run_ids.
Thanks for adding in the use of the $SCRATCH directory. I was on the fence on adding that in, but I think it makes sense to have with a note.
As for the example metadata file, I was planning on adding that in as part of working on #110 and will be sure that the file name matches the one that we used here.

Merging this so I can refer to it in docs more easily... I expect changes, but we know it works, at least partially (slurm mysteries remaining)


IDE-UI with Tree-Sitter causes uncaught TypeError (calling getScopeChain on undefined)

Prerequisites

[x] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Cannot reproduce in Safe Mode as it requires Atom-IDE-UI.

Description
While working on JavaScript code with Atom-IDE-UI and IDE-TypeScript packages installed, and enabling Tree-Sitter parsers, I get the following TypeError in the console every time I hover over a variable/function/other definition which triggers the datatip popup (video attached):
commitCommentView_item.graphql.js:112 Uncaught (in promise) TypeError: Cannot read property 'getScopeChain' of undefined
    at TextEditorRegistry.subscribeToSettingsForEditorScope (/Applications/Atom Beta.app/Contents/Resources/app/src/text-editor-registry.js:251:46)
    at TextEditorRegistry.updateAndMonitorEditorSettings (/Applications/Atom Beta.app/Contents/Resources/app/src/text-editor-registry.js:221:22)
    at <anonymous>
    at resolve (/Applications/Atom Beta.app/Contents/Resources/app.asar/node_modules/tree-sitter/index.js:296:21)
    at new Promise (<anonymous>)
    at Parser.parseTextBuffer (/Applications/Atom Beta.app/Contents/Resources/app.asar/node_modules/tree-sitter/index.js:295:10)
    at TreeSitterLanguageMode.parse (/Applications/Atom Beta.app/Contents/Resources/app/src/tree-sitter-language-mode.js:103:40)
    at LanguageLayer._performUpdate (/Applications/Atom Beta.app/Contents/Resources/app/src/tree-sitter-language-mode.js:565:48)
    at LanguageLayer.update (/Applications/Atom Beta.app/Contents/Resources/app/src/tree-sitter-language-mode.js:534:47)
    at new TreeSitterLanguageMode (/Applications/Atom Beta.app/Contents/Resources/app/src/tree-sitter-language-mode.js:75:34)
    at GrammarRegistry.languageModeForGrammarAndBuffer (/Applications/Atom Beta.app/Contents/Resources/app/src/grammar-registry.js:172:20)
    at TextEditor.setGrammar (/Applications/Atom Beta.app/Contents/Resources/app/src/text-editor.js:3816:48)
    at setupTextEditor (/Users/robert/.atom/packages/atom-ide-ui/modules/nuclide-commons-ui/AtomTextEditor.js:82:16)
    at AtomTextEditor.componentDidMount (/Users/robert/.atom/packages/atom-ide-ui/modules/nuclide-commons-ui/AtomTextEditor.js:113:28)
... (see attached file for full stack trace)

I am not seeing any effects of this error on actual functionality.
Steps to Reproduce

Install Atom-IDE-UI, IDE-TypeScript
Enable Tree-Sitter
Open DevTools
Open a JavaScript file and hover over a function or variable declaration/call site/something else which triggers a datatip
Observe error in the console

Expected behavior: No error in the console
Actual behavior: Error in the console
Reproduces how often: All the time
Versions
Atom    : 1.30.0-beta1
Electron: 2.0.5
Chrome  : 61.0.3163.100
Node    : 8.9.3

apm  1.19.0
npm  3.10.10
node 6.9.5 x64
atom 1.30.0-beta1
python 2.7.15
git 2.18.0

Additional Information
Full stack trace
Demo video (mov)

Thanks for taking the time to contribute!
We noticed that this is a duplicate of https://github.com/atom/atom/issues/17707. This has been fixed on master and the fix is available in the latest Atom nightly release :sparkles:
Because we treat our issues list as the Atom team's backlog, we close duplicates to focus our work and not have to touch the same chunk of code for the same reason multiple times. This is also why we may mark something as duplicate that isn't an exact duplicate but is closely related.
For information on how to use GitHub's search feature to find out if something is a duplicate before filing, see the How Can I Contribute? section of the Atom CONTRIBUTING guide.


Uses unique instance of GetIt locator to avoid conflicts with apps

GetIt supports creating individual instances. When using GetIt inside of a package this is good practice, otherwise conflicts can arise between an app using GetIt and the package.

Thanks!

Pushed in 0.2.3


Error in worker running for first time

How to reproduce
on branch unstable

Fresh installation or delete all data in bcdb/redis using "DO WITH CAUTION WILL DELETE ALL OF YOUR DATA"
bcdb delete --all
pkill -9 redis

Start 3bot server
kosmos -p "j.servers.threebot.local_start_explorer(with_shell=False, background=True)"

In tmux workers_1 or workers_2 you will find the following

Possible cause

Redis server replies with b'OK' at the first call

this should bever be b'OK'
where is this code, please next time also mention where screenshot is from
I found the code its in BCDBModelClient.get
this means it contacts the threebot server, is this the case?
is the threebot server active?
why is 'OK' coming back this should not be the case
there should be data coming back from the BCDB on the threebotserver
investigate more and let me know please

we found that the unwanted b'OK' response is coming from set method in RedisServer.py line 253:

after removing the new check and return the id in both cases it worked, but I'm not sure if that's a good fix.

I pushed the fix here: https://github.com/threefoldtech/jumpscaleX_core/commit/a1b383becd5f77046f43e87261b3039425e06718

verified on branch unstable