duration not working

Hi,
after upgrade to 2.0.4 and migrate my old config, the duration  (i.e. 3 hours) is not working correctly.
i have configured a 3 hours duration for a device, the starttime is looks good, but after one hour the planner switches off,,,
does anybody has similar issues?

Is it always like this, that it turn off after 1 hour?
In that case can you supply a summary of your current config (accept cost/rate + duration & search-length) and a screenshot with history data (planner-state and price) and even better point out in time when you believe it should keep on.

Maybe it‘s my fault, but i‘ve ported my config 1:1 from the configuration file to the new version, it could also be a bug (somewhere in the calculation i quess)
So here is my output

What would help in understanding what's happening is if you could add a screenshot of state history like this, with at least 23h (your search length) after the faulty behavior of binary sensor.

Should maybe clarify that setting the duration to 5h as in your example does not guarantee that you get 5h consecutive ON. It's just the number of consecutive hours to average when searching for the lowest average cost.
So let's as example say you have a 5h time-span during every night where the electricity price is much cheaper than the rest of the hours (and have a 23h search length), you would then find that spot during the whole day and wait for it to happen, but halfway or so into that period the next nights cheap span will start appearing in the horizon of your search and if this average turn out to be cheaper than your current 5h average (from current hour and 5h forward) it will turn off and wait for the next night.
So if you really want to achieve a 5h consecutive ON-period you likely need to add some automation to the binary switch to ensure that. (I'll think of if it might be something to add to the integration, but a bit tricky to keep history data in internal logic, similar as I'm struggling with the "static" planner #54 )

sure, here is a picture...

i've also tried a shorter search length, but the example was nearly the same. (i.e. 11 hours ... ) and i'm shure that this worked before in version 1.6.

well, i've reduced my search lengh to 11 hours.
don't get me wrong, the update is pretty cool, but i'am really confused, that an block of 3 hours worked better for my devices before the update.
maybe it would be cool, to add an option to use an coherent duration or an split duration. (just an idea)

but i'am really confused, that an block of 3 hours worked better for my devices before the update.

I don't think I have changed the expected design in that concern, but are some pretty large reworks in between v1 and v2. So either I have missed something in v2 (and if so I would like to know of it and fix), or it was actually a bug in v1 that worked in your favor but now is fixed.
If you can pin-point a specific time where you believe it should have stayed on, please share that history so that I can look in to the logic. (You should be able to add two integrations with different configurations)

maybe it would be cool, to add an option to use an coherent duration or an split duration. (just an idea)

Yes agree, I have some todo on that part, most of it is mentioned in #54, but are some concerns that I don't know howe to tackle.
Most of it relates to persistent storage so that the integration behaves predictable even if home assistant is restarted.
Have you tried tweaking the "accept_rate" config? If you have a price profile as in last picture your average price should be in the area of 0.30, setting an accept-rate of 0,5 should then turn on for every price below 0,15.

accept_rate is an good point i will try this.
meanwhile i‘ve purged an reinstalled the integration and my spooky problem seems to be gone, maybe just another ghost in the shell….


Samples: add multi-vehicle support

Added to vehicles sample in 1.0.17 and all samples starting with 1.0.18


Does HTTP_CF_CONNECTING_IP affect?

Hi
I added it in public/index.php
if (isset($_SERVER["HTTP_CF_CONNECTING_IP"])) {
    $_SERVER['REMOTE_ADDR'] = $_SERVER["HTTP_CF_CONNECTING_IP"];
}

Do I need to remove it?

I think this is not affect to this package

https://laravel-news.com/trusted-proxy


add condition to show count for more than one obs

closes #2081

@israr002 thanks for working on this. This approach is fine, but I think it would actually be better to add this conditional into the PhotoCount component (and update ObsImagePreview to remove the conditional there) so we can make sure we're showing the count consistently across the app. Does that make sense?

@albullington i have doubt here in the photoCount component there is a condition to add noevidence icon if there is no obs
and  we are not rendering photoCount if observation is less than or equal to 1 in obs media so if remove condition here then for 0 evidence it will show the noEvidence icon

But in the current app and web we are not showing that icon for obs without photos so shall i remove that piece of code ??


Bug fix: Using max_cov_count with integer typecast

Sorry for this bug.

I think it would be better to fix it by simply adding type=int rather than casting things afterwards:
    parser.add_argument('--max-cov-count', metavar='NUMBER', type=int,

ohh... that does sound better. Pardon my ignorance... new to python :) I will change it...

Sorry for late to the game. And thanks a lot for the fix!

@eddyxu Did you mean to merge this rather than close it? It looks like it got closed, but I don't see a change to coverage.py.

@myint Yea, my fault. Merged it.


[stable/traefik] Removed type: Opaque from ConfigMap

This PR fixes https://github.com/helm/charts/issues/21479
It's not necessary to use a Secret since CA certificate are suppose to be public.
This has been tested on my K3S node.
Thank you so much
@krancour @elsonrodriguez
Signed-off-by: Zhiqiang Wang<EMAIL_ADDRESS>What this PR does / why we need it:
There is a bug preventing enable m-tls function of traefik
Which issue this PR fixes

fixes https://github.com/helm/charts/issues/21479

Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

/assign @timoreimann


in aaLogWebAPI, the aalogDataSource.Instance singleton is not thread safe

Between null test and initialization of backing field there is a race condition where a second thread can test null and proceeding separately to initialize the backing field. The second thread would create a a second instance, which would replace the first. These two threads would then each have a separate instance of aaDataSource.

As we previously dicussed on Twitter, I will happily submit a pull request for this myself as a learning experience. I did some research to double-check the approach I was considering, but it turned out not to be as strong as I had originally thought. I'm thinking of using the Fifth version - fully lazy instantiation described in the article, Implementing the Singleton Pattern in C#.

I think the advice these days is to avoid singletons and let the dependency injection container manage your singletons for you. That said, I browsed your code and it's so elegant with the singleton. :-)

There is a bigger concern than the aalogDataSource singleton. Actually you can fix the singleton (the 6th version is definitely my preference these days when you can't use dependency injection), and there are other multi-threaded and multi-user issues.
The biggest issue is that the aaLogReader maintains all of the state information in the #region Globals section. So the FileStream, LogHeader, and all other module level variables are actually current state information for the last user or method call. I started to create multi-thread safe versions of some of the methods before realizing the multi-user concerns were there.
If multiple users are connected to aaLogWebAPI, when one user calls GetUnreadRecords it automatically changes the state information and affects the other user, specifically the LastRecordRead and possibly CurrentLogHeader if the logger moves to a new log file.
My original thought was to return the current LogRecord (last record read) along with the current LogFile information. But then we would need to add methods to check if the it LogFile has changed and then open a new FileStream. It starts to get messy.
I do not see an easy solution, other than storing the aaLogWebApi/aaLogReader state information for each connected user. aaLogReader seems fast enough that reopening the FileStream may not result in a big performance penalty.
The other option would be to create an aaLogPathManager class that is a singleton and reads the available log files and maintains the log file information such as first and last record and possibly associated time stamps. This would be to avoid repeated calls to aaLogReader.IndexLogHeaders. After that we would need to write a "stateless" and "stateful" version of aaLogReader. The "stateful" version would be much simpler to use for basic usage, and it would basically be a subclass of the stateless version with a built-in state. But if there are multi-threaded and multi-user concerns, then you need to use the stateless version.

Agree on all major points.  The original library didn't really have the concept of multi-user.
I also agree that the indexing and file streams are so fast I'm not sure it really matters.  So, with that in mind maybe we just do away with the singleton completely and create a new instance with each call?
Another idea would be to use the 'GetRecordsByMessageNumberAndCount call.  The client would track the last message number they received and when they call back they would pass this message number with the appropriate direction (forward) and count (if they want to page).  Once the count retrurned is less than the count they asked for they know they've got it all.
Thoughts?


wip websockets

Work In Progress. Initial refactor to support websockets.

It looks very nice and transparent, I'll check out the message queue, it looks good but I'm not super familiar. I think this is taking the right direction.

I have a lot of more work to go. So no rush it's going to change some more


🐞 - Date & DateRange & DateTime should accept single character date segment during paste

Which package(s) are the source of the bug?
@maskito/kit
Playground Link
https://maskito.dev/kit/date/API
Description
Case 1

https://maskito.dev/kit/date/API?mode=dd%2Fmm%2Fyyyy
Paste 11.9.2022

✅ Expected behaviour:
Textfield contains 11.09.2022.
❌ Actual behaviour:
Textfield contains 11.09.022.

Case 2

https://maskito.dev/kit/date/API?mode=dd%2Fmm%2Fyyyy
Paste 7.10.2020

✅ Expected behaviour:
Textfield contains 07.10.2020.
❌ Actual behaviour:
Textfield contains 07.02.020.
Maskito version
1.7.0
Which browsers have you used?

[X] Chrome
[ ] Firefox
[ ] Safari
[ ] Edge

Which operating systems have you used?

[X] macOS
[ ] Windows
[ ] Linux
[ ] iOS
[ ] Android

Relates to:

https://github.com/taiga-family/taiga-ui/issues/5615


quick fix to convert @required to required

When migrating code to null safety, I see error messages around @required parameters, in this case used in the definition of Flutter widgets:
The parameter 'title' can't have a value of 'null' because of its type, and no non-null default value is provided.
Try adding either a default value or the 'required' modifier.

The fix is to change the @required annotation to a required keyword. It would be great to offer a quick fix for this.
cc @pq @bwilkerson

We have a fix to add the required keyword. We might want to enhance it to remove the annotation (after ensuring that we're treating a required keyword as if it were a required annotation when analyzing code in opted out libraries).
We might also want a hint/lint to flag uses of the annotation in opted-in libraries.

Not that it's also possible that the fix isn't associated with the diagnostic shown above.

https://dart-review.googlesource.com/c/sdk/+/171420


Added support to pass custom networking context to snet

This allows apps to customize their networking context, e.g., by customizing the path resolver used by the context.

This change is

Reviewed 3 of 3 files at r1.
Review status: all files reviewed at latest revision, 2 unresolved discussions.

go/lib/snet/snet.go, line 103 at r1 (raw file):

// NewNetwork creates a minimal networking context without a path resolver.
// It is meant to be amended with a custom path resolver.
func NewNetwork(ia *addr.ISD_AS, sPath string, dPath string) *Network {

This breaks existing users of NewNetwork, if they exist.

go/lib/snet/snet.go, line 111 at r1 (raw file):

}

// NewDefaultNetwork creates a new networking context, on which future Dial or Listen

The name makes it sound like it assigns to DefNetwork. This is why i suggested leaving NewNetwork as-is, and adding NewNetworkBasic instead.

Comments from Reviewable

Review status: all files reviewed at latest revision, 2 unresolved discussions.

go/lib/snet/snet.go, line 103 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
This breaks existing users of NewNetwork, if they exist.

I like my naming scheme better and I don't think those users exist. Changed anyway.

go/lib/snet/snet.go, line 111 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
The name makes it sound like it assigns to DefNetwork. This is why i suggested leaving NewNetwork as-is, and adding NewNetworkBasic instead.

Done.

Comments from Reviewable

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


MediaQuery rebuilds widget when opening soft keyobard

First of all, the flutter team contributed such an excellent UI framework, he looks very good.
But recently I encountered some problems when using Scaffold's resizeToAvoidBottomInset property:
The resizeToAvoidBottomInset property is set to false in the sub-page, which seems to have no effect, but it is effective in the top-level page. For example, adding Scaffold in main.dart and setting resizeToAvoidBottomInset to false, but there is a problem in the sub-page. When the keyboard is awakened or hidden, the opened pages will be rebuilt, which greatly affects the performance of the APP, especially when the keyboard pops up or hides, do some animation, which looks a bit stuck。
Part of the code of main.dart：

Subpage part of the code：

Setting the attribute in main.dart is valid. If you delete the scaffold in main.dart, the subpage setting resizeToAvoidBottomInset to false will not have any effect, but if you set it in main.dart, when the keyboard pops up or hides, All your pages that have been opened will be rebuilt, which looks very bad
When the keyboard is awakened or hidden, the following happens

What should I do to make the resizeToAvoidBottomInset property only set and take effect in the subpages to prevent all pages from being rebuilt

I think wrapping your body with SingleChildScrollView with reverse: false/true, might work for you. Resizing will trigger rebuilds since the build context changes for the parents and thus has to be rebuilt.

Hi @ihongwu
Can you please provide your flutter doctor -v, a complete reproducible minimal code sample without using third-party and complete flutter run --verbose logs running the minimal code sample?
Thank you

I think wrapping your scaffold body with SingleChildScrollView with reverse: false/true, might work for you. Resizing will trigger rebuilds since the mediaquery size changes for the parents and thus has to be rebuilt. Also, are you using futurebuilder as the parent for any widget? That might also cause this issue

Thank you very much for your solution, but it does not seem to solve my problem. If I wrap Scaffold with SingleChildScrollView, when the keyboard pops up, the entire page will be pushed up, and I only want to animate the TextField input box upward, I think it will be rebuilt No problem, how to make it only rebuild the current page instead of rebuilding all pages, this is the goal I want to pursue

Can you provide a minimum reproducible example? I'll check on my end

@realrk95 @TahaTesser
Thank you very much for your answer. I think I have determined the problem. After setting resizeToAvoidBottomInset: false for Scaffold, in the future, if the page has the parameter MediaQuery.of(context).padding.bottom, you can even use MediaQuery.of( The size variable under context) will cause all the opened pages to be rebuilt. I temporarily replaced it with MediaQueryData.fromWindow(window).padding.bottom, and it seems that there is no problem.
But I still don't know why the use of all the size and size-related values under MediaQuery.of(context) will cause the page to be rebuilt.

This should not happen. Please share some example code of the issue. Mediaquery changes should not rebuild the parent widget.

Hi @ihongwu
To better address the issue, please provide the details

Can you please provide your flutter doctor -v, a complete reproducible minimal code sample without using third-party and complete flutter run --verbose logs running the minimal code sample?
Thank you

`
import 'package:flutter/material.dart';
void main() async{
runApp(new MyApp());

}
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
debugShowCheckedModeBanner: false,
title: 'MyApp',
home: MyHomePage(),
);
}
}
class MyHomePage extends StatefulWidget {
MyHomePage({Key ? key}) : super(key: key);
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State with SingleTickerProviderStateMixin{
PageController _pageController = PageController(initialPage: 0);
@override
Widget build(BuildContext context) {
return Scaffold(
resizeToAvoidBottomInset: false,
appBar: AppBar(
title: Text('Test Pages -'),
),
bottomNavigationBar: BottomNavigationBar(
onTap: (index){
_pageController.jumpToPage(index);
},
items: [
BottomNavigationBarItem(
icon: Icon(Icons.home),
label:'Home'
),
BottomNavigationBarItem(
icon: Icon(Icons.person_pin_circle),
label:'My'
),
],
),
body: PageView(
controller: _pageController,
children: [
Home1(),
My()
],
)
);
}
}
class Home1 extends StatefulWidget {
Home1({Key? key}) : super(key: key);
@override
_Home1State createState() => _Home1State();
}
class _Home1State extends State with AutomaticKeepAliveClientMixin{
@override
bool get wantKeepAlive => true;
@override
Widget build(BuildContext context) {
super.build(context);
print('Home Page build');
return Scaffold(
body: Container(
//  height: MediaQuery.of(context).size.height - 500,
child: Text('Use MediaQuery.of(context)'+MediaQuery.of(context).padding.bottom.toString()),
),
);
}
}
class My extends StatefulWidget {
My({Key? key}) : super(key: key);
@override
_MyState createState() => _MyState();
}
class _MyState extends State with AutomaticKeepAliveClientMixin{
@override
bool get wantKeepAlive => true;
@override
Widget build(BuildContext context) {
super.build(context);
print('My Page build');
return Scaffold(
body: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: [
Text('Use MediaQuery.of(context)'+MediaQuery.of(context).padding.bottom.toString()),
Container(height: 50,),
GestureDetector(
child: Text('Go to TextField Page',style: TextStyle(fontSize: 30),),
onTap: (){
Navigator.push(context, MaterialPageRoute(
builder: (context) => TextFieldPage(),
),
);
},
)
],
),
),
);
}
}
class TextFieldPage extends StatefulWidget {
TextFieldPage({Key? key}) : super(key: key);
@override
_TextFieldPageState createState() => _TextFieldPageState();
}
class _TextFieldPageState extends State {
@override
Widget build(BuildContext context) {
return Scaffold(
resizeToAvoidBottomInset: false,
body: Column(
children: [
Expanded(
child: ListView.builder(
reverse: true,
itemBuilder: (BuildContext context,int index){
return Text(index.toString());
}
)
),
TextField(
decoration: InputDecoration(
filled: true,
fillColor: Colors.blue[200],
hintText: 'input'
),
)
],
)
);
}
}
`
Click my page, then go to textfield page, and then click the input box to observe the rebuild

Hi @ihongwu
Thank for the code, I can reproduce the issue, I have created a simplest example to demonstrate this issue with just the home widget, it is not just a sub-page or nested page issue.
Run the code sample below, with Text uncomment and commented, home widget rebuilds when opening keyboard

minimal code sample
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Material App',
      theme: ThemeData.dark(),
      home: Home(),
    );
  }
}

class Home extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    print('Home>build');
    return Scaffold(
      resizeToAvoidBottomInset: false,
      appBar: AppBar(
        title: Text('Material App Bar'),
      ),
      body: Center(
        child: Column(
          children: [
            Container(
              /// Comment Text Widget below to prevent Home Widget from rebuilding
              child:  Text('Use MediaQuery.of(context)'+MediaQuery.of(context).padding.bottom.toString()),
            ),
            TextField(),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        child: Icon(Icons.add),
        onPressed: () {},
      ),
    );
  }
}

Expected Results
.F...... 0,0-1080,2148 aid=1073741824} flags=0 reason=SHOW_SOFT_INPUT
[  +56 ms] D/InsetsController( 5105): show(ime(), fromIme=true)
[  +16 ms] D/EGL_emulation( 5105): app_time_stats: avg=228.72ms min=1.55ms max=4336.45ms count=20

Actual Results
[  +81 ms] D/EGL_emulation( 5105): app_time_stats: avg=1415.19ms min=499.63ms max=2330.76ms count=2
[ +152 ms] D/InsetsController( 5105): show(ime(), fromIme=true)
[  +13 ms] D/EGL_emulation( 5105): app_time_stats: avg=8377.87ms min=2.06ms max=167281.72ms count=20
[  +74 ms] I/flutter ( 5105): Home>build
[  +83 ms] I/flutter ( 5105): Home>build
[  +38 ms] I/flutter ( 5105): Home>build
[  +59 ms] I/flutter ( 5105): Home>build
[  +41 ms] I/flutter ( 5105): Home>build
[  +31 ms] I/flutter ( 5105): Home>build
[ +926 ms] D/EGL_emulation( 5105): app_time_stats: avg=118.39ms min=29.33ms max=479.80ms count=12
[+1014 ms] D/EGL_emulation( 5105): app_time_stats: avg=507.67ms min=495.50ms max=519.84ms count=2
[+1481 ms] D/EGL_emulation( 5105): app_time_stats: avg=493.64ms min=480.84ms max=501.71ms count=3

stable
master

❌
❌

Check flutter doctor -v outputs for each channel below

flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on Microsoft Windows [Version 10.0.19043.1110], locale en-US)
    • Flutter version 2.2.3 at C:\Users\Taha\Code\flutter_stable
    • Framework revision f4abaa0735 (4 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\Taha\Code\android-sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_SDK_ROOT = C:\Users\Taha\Code\android-sdk
    • Java binary at: C:\Users\Taha\Code\android-studio\jre\bin\java.exe
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6842174)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[✓] Visual Studio - develop for Windows (Visual Studio Community 2019 16.10.4)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.10.31515.178
    • Windows 10 SDK version 10.0.19041.0

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/windows#android-setup for detailed instructions).

[✓] VS Code (version 1.58.2)
    • VS Code at C:\Users\Taha\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.24.0

[✓] Connected device (4 available)
    • sdk gphone64 x86 64 (mobile) • emulator-5554 • android-x64    • Android 11 (API 30) (emulator)
    • Windows (desktop)            • windows       • windows-x64    • Microsoft Windows [Version 10.0.19043.1110]
    • Chrome (web)                 • chrome        • web-javascript • Google Chrome 92.0.4515.107
    • Edge (web)                   • edge          • web-javascript • Microsoft Edge 91.0.864.71

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 2.4.0-5.0.pre.203, on Microsoft Windows [Version 10.0.19043.1110], locale en-US)
    • Flutter version 2.4.0-5.0.pre.203 at C:\Users\Taha\Code\flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 738ce43d97 (4 hours ago), 2021-07-26 23:40:49 -0700
    • Engine revision 717d55cdd4
    • Dart version 2.14.0 (build 2.14.0-354.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\Taha\Code\android-sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_SDK_ROOT = C:\Users\Taha\Code\android-sdk
    • Java binary at: C:\Users\Taha\Code\android-studio-preview\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[✓] Visual Studio - develop for Windows (Visual Studio Community 2019 16.10.4)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.10.31515.178
    • Windows 10 SDK version 10.0.19041.0

[✓] Android Studio (version 4.2)
    • Android Studio at C:\Users\Taha\Code\android-studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6842174)

[✓] Android Studio (version 2020.3)
    • Android Studio at C:\Users\Taha\Code\android-studio-preview
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code (version 1.58.2)
    • VS Code at C:\Users\Taha\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.24.0

[✓] Connected device (5 available)
    • sdk gphone64 x86 64 (mobile) • emulator-5554 • android-x64     • Android 11 (API 30) (emulator)
    • Windows (desktop)            • windows       • windows-x64     • Microsoft Windows [Version 10.0.19043.1110]
    • Windows (UWP) (desktop)      • winuwp        • windows-uwp-x64 •
    • Chrome (web)                 • chrome        • web-javascript  • Google Chrome 92.0.4515.107
    • Edge (web)                   • edge          • web-javascript  • Microsoft Edge 91.0.864.71

• No issues found!

✅: No Issue ❌: Issue reproduced

This is expected, when the keyboard comes up, the viewInsets and padding of the MediaQuery change, so all widgets depending on the MediaQuery need to rebuild to stay uptodate.


Remote Extension host terminated unexpectedly 3 times within the last 5 minutes.

Describe the bug
When working in "Dev Container" or "Remote connection" via ssh, the installation of the extension ends with an error:
Remote Extension host terminated unexpectedly 3 times within the last 5 minutes.
To Reproduce
Install extension, connect to remote, enable extension in remote environment
Logging
[2024-09-06T10:52:24.750Z] [10:52:24] Getting Manifest... rjmacarthy.twinny
[2024-09-06T10:52:24.913Z] [10:52:25] Installing extension: rjmacarthy.twinny {
  context: { clientTargetPlatform: 'win32-x64' },
  installOnlyNewlyAddedFromExtensionPack: true,
  isApplicationScoped: false,
  profileLocation: v {
    scheme: 'file',
    authority: '',
    path: '/home/node/.vscode-server/extensions/extensions.json',
    query: '',
    fragment: '',
    _formatted: 'file:///home/node/.vscode-server/extensions/extensions.json',
    _fsPath: '/home/node/.vscode-server/extensions/extensions.json'
  },
  productVersion: { version: '1.92.2', date: '2024-08-14T17:29:30.058Z' }
}
[2024-09-06T10:52:35.558Z] [10:52:35] Extension signature verification result for rjmacarthy.twinny: Success. Executed: true. Duration: 2027ms.
[2024-09-06T10:52:38.963Z] [10:52:39] Extracted extension to file:///home/node/.vscode-server/extensions/rjmacarthy.twinny-3.15.13-linux-x64: rjmacarthy.twinny
[2024-09-06T10:52:38.974Z] [10:52:39] Renamed to /home/node/.vscode-server/extensions/rjmacarthy.twinny-3.15.13-linux-x64
[2024-09-06T10:52:38.991Z] [10:52:39] Extension installed successfully: rjmacarthy.twinny file:///home/node/.vscode-server/extensions/extensions.json
[2024-09-06T10:52:52.620Z] Port forwarding connection from 16266 > 42801 > 42801 in the container.
[2024-09-06T10:52:52.621Z] Start: Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:52.768Z] Port forwarding 16266 > 42801 > 42801 stderr: Connection established
[2024-09-06T10:52:53.060Z] [10:52:53] [<IP_ADDRESS>][d1bda003][ExtensionHostConnection] <412> Extension Host Process exited with code: null, signal: SIGILL.
[2024-09-06T10:52:53.105Z] Port forwarding 16204 > 42801 > 42801 stderr: Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:218:20)
[2024-09-06T10:52:53.106Z] Port forwarding 16204 > 42801 > 42801 stderr: Remote close with error
[2024-09-06T10:52:53.106Z] Stop (106837 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:53.107Z] Port forwarding 16204 > 42801 > 42801 terminated with code 1 and signal null.
[2024-09-06T10:52:53.109Z] Port forwarding 16204 > 42801 > 42801: Local close
[2024-09-06T10:52:53.122Z] Port forwarding connection from 16269 > 42801 > 42801 in the container.
[2024-09-06T10:52:53.122Z] Start: Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:53.261Z] Port forwarding 16269 > 42801 > 42801 stderr: Connection established
[2024-09-06T10:52:53.323Z] [10:52:53] [<IP_ADDRESS>][d1bda003][ExtensionHostConnection] Unknown reconnection token (seen before).
[2024-09-06T10:52:53.366Z] Port forwarding 16269 > 42801 > 42801 stderr: Remote close
[2024-09-06T10:52:53.367Z] Stop (245 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:53.368Z] Port forwarding 16269 > 42801 > 42801 terminated with code 0 and signal null.
[2024-09-06T10:52:53.371Z] Port forwarding 16269 > 42801 > 42801: Local close
[2024-09-06T10:52:53.410Z] Port forwarding connection from 16270 > 42801 > 42801 in the container.
[2024-09-06T10:52:53.411Z] Start: Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:53.538Z] Port forwarding 16270 > 42801 > 42801 stderr: Connection established
[2024-09-06T10:52:53.605Z] [10:52:53] [<IP_ADDRESS>][0705cdf2][ExtensionHostConnection] New connection established.
[2024-09-06T10:52:53.649Z] [10:52:53] [<IP_ADDRESS>][0705cdf2][ExtensionHostConnection] <1020> Launched Extension Host Process.
[2024-09-06T10:52:55.878Z] [10:52:55] [<IP_ADDRESS>][0705cdf2][ExtensionHostConnection] <1020> Extension Host Process exited with code: null, signal: SIGILL.
[2024-09-06T10:52:55.921Z] Port forwarding 16270 > 42801 > 42801 stderr: Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:218:20)
[2024-09-06T10:52:55.921Z] Port forwarding 16270 > 42801 > 42801 stderr: Remote close with error
[2024-09-06T10:52:55.922Z] Stop (2511 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:55.922Z] Port forwarding 16270 > 42801 > 42801 terminated with code 1 and signal null.
[2024-09-06T10:52:55.924Z] Port forwarding 16270 > 42801 > 42801: Local close
[2024-09-06T10:52:55.928Z] Port forwarding connection from 16272 > 42801 > 42801 in the container.
[2024-09-06T10:52:55.929Z] Start: Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:56.053Z] Port forwarding 16272 > 42801 > 42801 stderr: Connection established
[2024-09-06T10:52:56.107Z] [10:52:56] [<IP_ADDRESS>][0705cdf2][ExtensionHostConnection] Unknown reconnection token (seen before).
[2024-09-06T10:52:56.149Z] Port forwarding 16272 > 42801 > 42801 stderr: Remote close
[2024-09-06T10:52:56.150Z] Stop (221 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:56.151Z] Port forwarding 16272 > 42801 > 42801 terminated with code 0 and signal null.
[2024-09-06T10:52:56.152Z] Port forwarding 16272 > 42801 > 42801: Local close
[2024-09-06T10:52:56.188Z] Port forwarding connection from 16273 > 42801 > 42801 in the container.
[2024-09-06T10:52:56.189Z] Start: Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:56.330Z] Port forwarding 16273 > 42801 > 42801 stderr: Connection established
[2024-09-06T10:52:56.388Z] [10:52:56] [<IP_ADDRESS>][8ab1e03b][ExtensionHostConnection] New connection established.
[2024-09-06T10:52:56.433Z] [10:52:56] [<IP_ADDRESS>][8ab1e03b][ExtensionHostConnection] <1117> Launched Extension Host Process.
[2024-09-06T10:52:57.774Z] Port forwarding 16266 > 42801 > 42801 stderr: Remote close
[2024-09-06T10:52:57.818Z] Stop (5197 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:57.819Z] Port forwarding 16266 > 42801 > 42801 terminated with code 0 and signal null.
[2024-09-06T10:52:58.917Z] [10:52:59] [<IP_ADDRESS>][8ab1e03b][ExtensionHostConnection] <1117> Extension Host Process exited with code: null, signal: SIGILL.
[2024-09-06T10:52:58.961Z] Port forwarding 16273 > 42801 > 42801 stderr: Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:218:20)
[2024-09-06T10:52:58.961Z] Port forwarding 16273 > 42801 > 42801 stderr: Remote close with error
[2024-09-06T10:52:58.962Z] Stop (2773 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:58.962Z] Port forwarding 16273 > 42801 > 42801 terminated with code 1 and signal null.
[2024-09-06T10:52:58.964Z] Port forwarding 16273 > 42801 > 42801: Local close
[2024-09-06T10:52:58.974Z] Port forwarding connection from 16274 > 42801 > 42801 in the container.
[2024-09-06T10:52:58.975Z] Start: Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:59.103Z] Port forwarding 16274 > 42801 > 42801 stderr: Connection established
[2024-09-06T10:52:59.163Z] [10:52:59] [<IP_ADDRESS>][8ab1e03b][ExtensionHostConnection] Unknown reconnection token (seen before).
[2024-09-06T10:52:59.206Z] Port forwarding 16274 > 42801 > 42801 stderr: Remote close
[2024-09-06T10:52:59.206Z] Stop (231 ms): Run in container: /home/node/.vscode-server/bin/fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/node -e
[2024-09-06T10:52:59.207Z] Port forwarding 16274 > 42801 > 42801 terminated with code 0 and signal null.
[2024-09-06T10:52:59.208Z] Port forwarding 16274 > 42801 > 42801: Local close

Desktop (please complete the following information):

OS (local): windows 10 22h2 19045.4780
Browser [Edge 128.0.2739.54]
OS (remote): debian 12 (console only), docker 24.0.9
DevContainer: "image": "mcr.microsoft.com/devcontainers/typescript-node:1-20-bookworm"

Hey it seems you're client target is windows context: { clientTargetPlatform: 'win32-x64' } but the container is using linux but the profileLocation is using linux x64?.

Well, I work on windows, so probably win32-x64 comes from there. And the remote machine on debian 12 x64...
The extension works locally on Windows, but when I use "Open Remote Window" via "Remote - SSH" and install the local extension in 'SSH:remotehost', the extension host process on the remote machine crashes.
All logs from the windows user profile and the debian user's home directory are attached.
P.S.: The same thing happens when I use "Dev Container"
codelogs.zip


Remove SSLv3 usage

Test and migrate to TLS to Jembi

Other than the question 'bout the version changes, looks good.

:+1:


pipenv shell fails with msys

pipenv shell fails in an msys2 shell:
../Python36/Scripts/pipenv shell                                                              │
zsh:1: command not found: c:python36python.exe

Please run $ python -m pipenv.help, and paste the results here.
$ python -m pipenv.help output
Pipenv version: '11.10.0'
Pipenv location: 'C:\\Python36\\lib\\site-packages\\pipenv'
Python location: 'C:\\Python36\\python.exe'
Other Python installations in PATH:

2.7: C:\msys64\usr\bin\python2.7.exe

2.7: C:\msys64\usr\bin\python2.7.exe

3.6: C:\msys64\mingw64\bin\python3.6m.exe

3.6: C:\msys64\mingw64\bin\python3.6.exe

3.6: C:\msys64\usr\bin\python3.6.exe

3.6.2: C:\msys64\usr\bin\python.exe

2.7.13: C:\msys64\usr\bin\python2.exe

3.6.4: C:\msys64\mingw64\bin\python3.exe

3.6.2: C:\msys64\usr\bin\python3.exe

3.6.5: C:\Windows\py.exe

PEP 508 Information:
{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '10',
 'platform_system': 'Windows',
 'platform_version': '10.0.16299',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'win32'}

System environment variables:

!;

Actual result
It appears that the wrong path separators are used?
Steps to replicate

Installed msys2
Using zsh
Installed Python 3.6 from Python.org
Installed the pipenv package

Pipenv doesn't support msys on windows (officially)
Use pipenv run instead
We use the correct path separators because we don't do checks for if you use linux emulation on windows
This might work if you used ../Python36/Scripts/python -m pipenv shell but I have no idea

Our software stack is super complicated already since we handle random stuff like this, in addition to dependency management/environment management/dependency resolution across platforms and python versions, which is why we kind of draw the line there. The overhead of maintaining the codebase so it works not just across these platforms but in addition has to recognize when it runs in a partial emulation implementation is too much overhead for a development team of our size.
That said, if someone has a simple fix for this I'm sure we'd be willing to review it
For reference we are just using python's separators here I believe.  I'm gonna close this for now since fixing it is not on the roadmap and isn't likely to make it on a priority list anywhere, FWIW I'd just use pipenv run (I do this myself, I don't really ever use pipenv shell in my workflow)


Update Move-SPProfileManagedMetadataProperty.md

Remove redundant example. Update existing example to be more clear.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sharepoint/sharepoint-ps/sharepoint-server/Move-SPProfileManagedMetadataProperty.md
:white_check_mark:Succeeded
View (sharepoint-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add emergency keywords that take users to the please call me flow

SXD has provided us with a sheet with the emergency keywords that need to be supported. I have included the csv exported, so that it is easy for us to update based on the sheet SXD is using.
After chatting with Chuk, they would like us to accommodate for variations of these words/phrases. I spoke to Rudi about fuzzy search and then implemented a fuzzy search with a cutoff score of 75, this unfortunately does catch some other keywords, which makes me nervous, but the check is done first, so if those keywords are in other lists, they will be overwritten. Where this really makes me nervous is misspellings of the other keywords might incorrectly categorize a keyword.
I think there are three solutions to this,

tighten the acceptable score and miss out on catching phrase variations and only catch misspellings
request a significantly more substantial list of phrases from SXD and tighten the scoring
implement fuzzy search for all the keywords (I think this is a lovely idea, but I dont know how many problems it may cause, but I'd be happy to investigate as it will account for any typos users send in)

I have also tabulated the matches below for a positive match on a score of 75 or above

keyword
match
match ratio

hi
hit
80.0

depressed
depressed
100.0

i want to die
i want to die
100.0

edpressed
depressed
88.9

i am depressed
depressed
78.3

help
help
100.0

hi
hit
80.0

heita
hit
75.0

For a score of 85 and above, which catches typos and misses phrases unless they are in the csv:

keyword
match
match ratio

depressed
depressed
100.0

i want to die
i want to die
100.0

edpressed
depressed
88.9

help
help
100.0

Resources:

ticket: https://praekeltorg.atlassian.net/browse/YAL-394
sheet: https://docs.google.com/spreadsheets/d/1MpKmtDdFvc1nlF7Eu37pIKCa59DKqOR5Pr_gJcVjsyQ/edit#gid=0

@megh-k was there any conversation with SXD about making this less disruptive? like sending an additional message with instructions to get to the counselling flow instead of sending them directly there?

@KaitCrawford I asked about whether some keywords weren't too broad but the conclusion was "lets see how it goes first" but I will check in again regarding an additional step

@megh-k maybe a compromise would be that exact matches get sent there directly but fuzzy matches just get something like "are you ok? send in help to speak to a counsellor"


Add a Coordinape User in all circles

Summary
Add a Coordinape User to all circles, with a fixed address - to allow the protocol to receive gifts from users.
Goes with FE: https://github.com/coordinape/coordinape/pull/235
https://linear.app/yearn/issue/APE-305/create-a-coordinape-user
> docker exec -it app php artisan ape:populate_coordinate_user ADDRESS

Based off the staging branch - because that's what contains the new Circle Create API: https://github.com/coordinape/coordinape-backend/pull/47
Questions:

so it looks like we should be creating one (1) new profile record for the Coordinape user, and many (n) new users  records for this user in each circle. This seems like it will duplicate data but I think that's ok for now

TODOs:

[x] Create one profile with a HARDCODED (from .env) Coordinape eth address
[x] Create users entries for this user in all circles, adding Coordinape to each circle (with task)
[x] Add this user to all new circles too
[ ] FE changes in a separate PR: https://github.com/coordinape/coordinape/pull/235

Deploy Instructions

Merge FE PR
Merge this PR
Run php artisan ape:populate_coordinape_user

@zashton so any ideas on best way to lock down this new user? Stop anyone including admins from changing its properties? We would only want to let admins remove/re-add the user to a circle?

@crabsinger you can disable/hide this user on the FE admin dashboard and then on the backend block this user type from being updated. We do not have a super admin dashboard, so if you need to you can soft delete the user from the db.

@zashton PTAL. I've updated most of the outstanding points. One remaining uqestion on the IsCoordinapeUser accessor, as it was erroring for me.
Let me know what else you see before this is ready for merge.

@zashton PTAL. I've updated most of the outstanding points. One remaining uqestion on the IsCoordinapeUser accessor, as it was erroring for me.
Let me know what else you see before this is ready for merge.

i assume you are using it like $user->isCoordinapeUser()  ? instead use it like $user->isCoordinapeUser

@zashton, pushed the changes to the accessor method. Looks like its working now. 👍
If we want to update the Coordinape User address right now we'd need to do:

change ENV variable (this will make new users have this new address)
update all old users's addresses to the new address.

I guess we could create a helper method to update all the old addresses... or just do it SQL? Thoughts?

:shipit:


Dependency versions issue

I am having issues with getting hcaptcha to work with the dependency packages. If I downgrade react-native-webview to 9.0.1 then I have to downgrade react verison, react native verison and other packages that break.
[stderr] npm ERR! Found<EMAIL_ADDRESS>[stderr] npm ERR! node_modules/react-native-webview
[stderr] npm ERR<EMAIL_ADDRESS>from the root project
[stderr] npm ERR!
[stderr] npm ERR! Could not resolve dependency:
[stderr] npm ERR! peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>[stderr] npm ERR! node_modules/@hcaptcha/react-native-hcaptcha
[stderr] npm ERR<EMAIL_ADDRESS>from the root project
My package json:
"@expo/vector-icons": "^12.0.5",
"@hcaptcha/react-native-hcaptcha": "^1.3.4",
"@react-native-async-storage/async-storage": "~1.15.0",
"@react-native-community/clipboard": "^1.5.1",
"@react-native-community/masked-view": "^0.1.11",
"@react-native-firebase/app": "^14.5.0",
"@react-navigation/drawer": "^5.12.5",
"@react-navigation/material-top-tabs": "^5.3.15",
"@react-navigation/native": "^5.9.4",
"@react-navigation/stack": "^5.14.5",
"axios": "^0.26.0",
"expo": "~44.0.0",
"expo-app-loading": "^1.3.0",
"expo-auth-session": "~3.5.0",
"expo-dev-client": "^0.8.4",
"expo-linear-gradient": "~11.0.3",
"expo-localization": "~12.0.0",
"expo-notifications": "~0.14.0",
"expo-random": "~12.1.1",
"expo-splash-screen": "~0.14.1",
"expo-status-bar": "~1.2.0",
"expo-updates": "^0.11.6",
"formik": "^2.2.9",
"i18n-js": "^3.8.0",
"react": "17.0.1",
"react-dom": "17.0.1",
"react-native": "0.64.3",
"react-native-app-intro-slider": "^4.0.4",
"react-native-applovin-max": "^2.5.2",
"react-native-collapsible": "^1.6.0",
"react-native-confirmation-code-input": "^1.0.4",
"react-native-dotenv": "^3.3.1",
"react-native-elements": "^3.4.2",
"react-native-fabric": "^0.5.2",
"react-native-gesture-handler": "~2.1.0",
"react-native-modal": "^11.5.4",
"react-native-pager-view": "5.4.9",
"react-native-reanimated": "^2.3.1",
"react-native-safe-area-context": "3.3.2",
"react-native-screens": "~3.10.1",
"react-native-tab-view": "^3.1.1",
"react-native-vector-icons": "^9.1.0",
"react-native-web": "0.17.1",
"react-native-webview": "^11.0.0",
"styled-components": "^5.3.3"

@CAMOBAP please take a look.


CI: GuestBook Examples checks policy example: Cannot get web pods

https://jenkins.cilium.io/job/cilium-ginkgo/job/cilium/job/master/5573/testReport/junit/Suite-k8s-1/12/K8sPolicyTest_GuestBook_Examples_checks_policy_example/
e60a837a_K8sPolicyTest_GuestBook_Examples_checks_policy_example.zip
Observed on master branch, v1.7 branch; coming up on 40 failures today. Seems like something in guestbook is either reliant on external infrastructure that is no longer available or otherwise configured with some kind of timebomb (expired cert, etc.).
Stacktrace
/home/jenkins/workspace/cilium-ginkgo_cilium_master/k8s-1.12-gopath/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:514
Cannot get web pods
Expected
    <*errors.errorString | 0xc001b9ebc0>: {
        s: "cannot retrieve pods: error: error executing jsonpath \"{range .items[*]}{<EMAIL_ADDRESS>Error executing template: not in range, nothing to end. Printing more information for debugging the template:\n\ttemplate was:\n\t\t{range .items[*]}{<EMAIL_ADDRESS>given to jsonpath engine was:\n\t\tmap[string]interface {}{\"apiVersion\":\"v1\", \"items\":[]interface {}{}, \"kind\":\"List\", \"metadata\":map[string]interface {}{\"resourceVersion\":\"\", \"selfLink\":\"\"}}\n\n\n",
    }
to be nil
/home/jenkins/workspace/cilium-ginkgo_cilium_master/k8s-1.12-gopath/src/github.com/cilium/cilium/test/k8sT/Policies.go:1585

Standard Output
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Cilium pods: [cilium-ctcs5 cilium-mhwk6]
Netpols loaded: default::guestbook-policy-redis
CiliumNetworkPolicies loaded: default::guestbook-policy-web
Endpoint Policy Enforcement:
Pod                           Ingress   Egress
frontend-c749f689b-m92c8
redis-master-t8r9v
redis-slave-9mltw
coredns-5544f57546-ts98r
grafana-559c9666fd-ngbwd
prometheus-7b5bd859df-k4m7c
Cilium agent 'cilium-ctcs5': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 27 Failed 0
Cilium agent 'cilium-mhwk6': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 26 Failed 0

Standard Error
20:35:07 STEP: Running BeforeEach block for EntireTestsuite K8sPolicyTest GuestBook Examples
20:35:08 STEP: WaitforPods(namespace="default", filter="-l zgroup=guestbook")
20:35:13 STEP: WaitforPods(namespace="default", filter="-l zgroup=guestbook") => <nil>
20:35:13 STEP: Apply policy to web
20:35:20 STEP: Apply policy to Redis
FAIL: Cannot get web pods
Expected
    <*errors.errorString | 0xc001b9ebc0>: {
        s: "cannot retrieve pods: error: error executing jsonpath \"{range .items[*]}{<EMAIL_ADDRESS>Error executing template: not in range, nothing to end. Printing more information for debugging the template:\n\ttemplate was:\n\t\t{range .items[*]}{<EMAIL_ADDRESS>given to jsonpath engine was:\n\t\tmap[string]interface {}{\"apiVersion\":\"v1\", \"items\":[]interface {}{}, \"kind\":\"List\", \"metadata\":map[string]interface {}{\"resourceVersion\":\"\", \"selfLink\":\"\"}}\n\n\n",
    }
to be nil
=== Test Finished at 2020-08-27T20:35:21Z====
20:35:21 STEP: Running JustAfterEach block for EntireTestsuite K8sPolicyTest
===================== TEST FAILED =====================
20:35:22 STEP: Running AfterFailed block for EntireTestsuite K8sPolicyTest
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0
Stdout:
 	 NAMESPACE           NAME                              READY   STATUS    RESTARTS   AGE     IP              NODE   NOMINATED NODE
	 cilium-monitoring   grafana-559c9666fd-ngbwd          0/1     Running   0          100m    <IP_ADDRESS>       k8s2   <none>
	 cilium-monitoring   prometheus-7b5bd859df-k4m7c       1/1     Running   0          100m    <IP_ADDRESS>      k8s2   <none>
	 default             frontend-c749f689b-m92c8          1/1     Running   0          17s     <IP_ADDRESS>       k8s2   <none>
	 default             redis-master-t8r9v                1/1     Running   0          17s     <IP_ADDRESS>      k8s1   <none>
	 default             redis-slave-9mltw                 1/1     Running   0          17s     <IP_ADDRESS>      k8s1   <none>
	 kube-system         cilium-ctcs5                      1/1     Running   0          2m10s   <IP_ADDRESS>   k8s2   <none>
	 kube-system         cilium-mhwk6                      1/1     Running   0          2m10s   <IP_ADDRESS>   k8s1   <none>
	 kube-system         cilium-operator-fccff88f7-n95vq   1/1     Running   0          2m10s   <IP_ADDRESS>   k8s3   <none>
	 kube-system         coredns-5544f57546-ts98r          1/1     Running   0          30m     <IP_ADDRESS>       k8s2   <none>
	 kube-system         etcd-k8s1                         1/1     Running   0          107m    <IP_ADDRESS>   k8s1   <none>
	 kube-system         kube-apiserver-k8s1               1/1     Running   0          107m    <IP_ADDRESS>   k8s1   <none>
	 kube-system         kube-controller-manager-k8s1      1/1     Running   0          108m    <IP_ADDRESS>   k8s1   <none>
	 kube-system         kube-scheduler-k8s1               1/1     Running   0          108m    <IP_ADDRESS>   k8s1   <none>
	 kube-system         log-gatherer-bwgjt                1/1     Running   0          100m    <IP_ADDRESS>   k8s3   <none>
	 kube-system         log-gatherer-fvpzp                1/1     Running   0          100m    <IP_ADDRESS>   k8s1   <none>
	 kube-system         log-gatherer-l2kzh                1/1     Running   0          100m    <IP_ADDRESS>   k8s2   <none>
	 kube-system         registry-adder-k7xpv              1/1     Running   0          106m    <IP_ADDRESS>   k8s2   <none>
	 kube-system         registry-adder-vd8tp              1/1     Running   0          106m    <IP_ADDRESS>   k8s1   <none>
	 kube-system         registry-adder-z8kzb              1/1     Running   0          106m    <IP_ADDRESS>   k8s3   <none>

Stderr:

Fetching command output from pods [cilium-ctcs5 cilium-mhwk6]
cmd: kubectl exec -n kube-system cilium-ctcs5 -- cilium service list
Exitcode: 0
Stdout:
 	 ID   Frontend              Service Type   Backend
	 1    <IP_ADDRESS>:3000    ClusterIP
	 2    <IP_ADDRESS>:9090   ClusterIP      1 => <IP_ADDRESS>:9090
	 3    <IP_ADDRESS>:443         ClusterIP      1 => <IP_ADDRESS>:6443
	 4    <IP_ADDRESS>:53         ClusterIP      1 => <IP_ADDRESS>:53
	 95   <IP_ADDRESS>:6379    ClusterIP      1 => <IP_ADDRESS>:6379
	 96   <IP_ADDRESS>:6379   ClusterIP      1 => <IP_ADDRESS>:6379

Stderr:

cmd: kubectl exec -n kube-system cilium-ctcs5 -- cilium endpoint list
Exitcode: 0
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                       IPv6       IPv4        STATUS
	            ENFORCEMENT        ENFORCEMENT
	 560        Disabled           Disabled          51634      k8s:io.cilium.k8s.policy.cluster=default          fd00::66   <IP_ADDRESS>   ready
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns
	                                                            k8s:io.kubernetes.pod.namespace=kube-system
	                                                            k8s:k8s-app=kube-dns
	 1521       Enabled            Disabled          17289      k8s:app=guestbook                                 fd00::7e   <IP_ADDRESS>   ready
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:tier=frontend
	 2147       Disabled           Disabled          4          reserved:health                                   fd00::46   <IP_ADDRESS>   ready
	 3986       Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s2                                               ready
	                                                            reserved:host

Stderr:

cmd: kubectl exec -n kube-system cilium-mhwk6 -- cilium service list
Exitcode: 0
Stdout:
 	 ID   Frontend              Service Type   Backend
	 1    <IP_ADDRESS>:443         ClusterIP      1 => <IP_ADDRESS>:6443
	 2    <IP_ADDRESS>:53         ClusterIP      1 => <IP_ADDRESS>:53
	 3    <IP_ADDRESS>:3000    ClusterIP
	 4    <IP_ADDRESS>:9090   ClusterIP      1 => <IP_ADDRESS>:9090
	 86   <IP_ADDRESS>:40000   HostPort       1 => <IP_ADDRESS>:8080
	 87   <IP_ADDRESS>:40000       HostPort       1 => <IP_ADDRESS>:8080
	 88   <IP_ADDRESS>:40000         HostPort       1 => <IP_ADDRESS>:8080
	 89   <IP_ADDRESS>:6379    ClusterIP      1 => <IP_ADDRESS>:6379
	 90   <IP_ADDRESS>:6379   ClusterIP      1 => <IP_ADDRESS>:6379

Stderr:

cmd: kubectl exec -n kube-system cilium-mhwk6 -- cilium endpoint list
Exitcode: 0
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                       IPv6        IPv4         STATUS
	            ENFORCEMENT        ENFORCEMENT
	 220        Enabled            Disabled          50389      k8s:io.cilium.k8s.policy.cluster=default          fd00::1bf   <IP_ADDRESS>   ready
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:k8s-app.guestbook=redis
	                                                            k8s:role=master
	                                                            k8s:zgroup=guestbook
	 539        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s1                                                 ready
	                                                            k8s:node-role.kubernetes.io/master
	                                                            reserved:host
	 862        Disabled           Disabled          4          reserved:health                                   fd00::1fb   <IP_ADDRESS>   ready
	 1255       Enabled            Disabled          42266      k8s:io.cilium.k8s.policy.cluster=default          fd00::18e   <IP_ADDRESS>   ready
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:k8s-app.guestbook=redis
	                                                            k8s:role=slave
	                                                            k8s:zgroup=guestbook

Stderr:

===================== Exiting AfterFailed =====================
20:36:02 STEP: Running AfterEach for block EntireTestsuite K8sPolicyTest GuestBook Examples
FAIL: Redis deprecated policy cannot be deleted
Expected command: kubectl delete -f  /home/jenkins/workspace/cilium-ginkgo_cilium_master/k8s-1.12-gopath/src/github.com/cilium/cilium/test/k8sT/manifests/guestbook-policy-redis-deprecated.json
To succeed, but it failed:
Exitcode: 1
Err: exit status 1
Stdout:

Stderr:
 	 Error from server (NotFound): error when deleting "/home/jenkins/workspace/cilium-ginkgo_cilium_master/k8s-1.12-gopath/src/github.com/cilium/cilium/test/k8sT/manifests/guestbook-policy-redis-deprecated.json": networkpolicies.extensions "guestbook-redis-deprecated" not found

20:36:07 STEP: Running AfterEach for block EntireTestsuite K8sPolicyTest
20:36:07 STEP: Running AfterEach for block EntireTestsuite

I think that #13003 will fix this flake. The relation may not be obvious, but the error message seems to indicate that the jsonpath is failing because of the use of {end} on what is effectively an empty list. #13003 will make sure the list of items (selected pods) isn't null.
s: "cannot retrieve pods: error: error executing jsonpath "{range .items[*]}{<EMAIL_ADDRESS>Error executing template: not in range, nothing to end. Printing more information for debugging the template:
	template was:
		{range .items[*]}{<EMAIL_ADDRESS>	object given to jsonpath engine was:
		map[string]interface {}{"apiVersion":"v1", "items":[]interface {}{}, "kind":"List", "metadata":map[string]interface {}{"resourceVersion":"", "selfLink":""}}",

It's not clear that a code change on our side introduced this bug. It started failing in the v1.8 and v1.7 branches on the 27th of August: https://datastudio.google.com/s/nXAOxbdZN_I. The one v1.8 backport PR on those dates (https://github.com/cilium/cilium/pull/12963 from https://github.com/cilium/cilium/issues?q=label%3Abackport%2F1.8+merged%3A2020-08-25..2020-08-27+is%3Aclosed+) doesn't appear to have any related changes. Same for the only v1.7 backport PR (https://github.com/cilium/cilium/pull/12966 from https://github.com/cilium/cilium/issues?q=label%3Abackport%2F1.7+merged%3A2020-08-25..2020-08-27+is%3Aclosed+).
We likely have some external dependency that introduced this bug in our tests.


Model file not found

While running "python app/flask_api.py" this command, I get this error how can I get rid of this error?
return self._float_to_str(self.smallest_subnormal)
Traceback (most recent call last):
File "/home/shihab/Projects/indicwav2back/IndicWav2Vec/ULCA_Compliance/app/support.py", line 238, in load_model
mdl, cfg, task = fairseq.checkpoint_utils.load_model_ensemble_and_task([mdl_path])
File "/home/shihab/Projects/indicwav2back/IndicWav2Vec/venv/lib/python3.8/site-packages/fairseq/checkpoint_utils.py", line 278, in load_model_ensemble_and_task
raise IOError("Model file not found: {}".format(filename))
OSError: Model file not found: /new_disk/models/bn/bn.pt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "app/flask_api.py", line 28, in
model, dictionary = load_model(m['model_path'])
File "/home/shihab/Projects/indicwav2back/IndicWav2Vec/ULCA_Compliance/app/support.py", line 243, in load_model
m = torch.load(mdl_path)
File "/home/shihab/Projects/indicwav2back/IndicWav2Vec/venv/lib/python3.8/site-packages/torch/serialization.py", line 594, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/shihab/Projects/indicwav2back/IndicWav2Vec/venv/lib/python3.8/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/shihab/Projects/indicwav2back/IndicWav2Vec/venv/lib/python3.8/site-packages/torch/serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/new_disk/models/bn/bn.pt'

Hi [shihabtechno]
There should be a json file containing paths to your model checkpoints. You will need to update your model paths there.


Create Gantt diagram

Create a Gantt diagram where the project is decomposed into smaller tasks and define dependencies between them. Task already made should also be included, e.g.: research of Identity, Proofs & Claims, Authentication.

I've made a first draft of the diagram using ProjectLibre. You can download it here: https://www.projectlibre.com/product/projectlibre-open-source

The end date is 9th of July which exceeds the Decentralized Web Summit dates, but:

There are styleguide related tasks that might less than 1 day
I only added myself as a resource but if @marcooliveira does some tasks, we can deliver earlier

As a note, I've not added the search + invite via blockstack id as a task because we won't have time to do it for the POC. Neverthless I've added an issue for it.

You put "July", when you meant "August". I fixed the comment.


feat(cli): migrate default interactive - optional ci

This PR makes --interactive the default and removes the need for the flag. Additionally it adds a new flag --ci which is the inverse of interactive.
@chadhietala --ci will need to be updated for all automated scripts / actions

👍 but I think we need to remove --ci as well in the future and just pass all required answers to any interactive questions as CLI options.

@egorio yes and/or from the config


Reward screen after completed shooting round

[ ] Physical scoreboard
[ ] Camera pan to scoreboard at the end of the round
[ ] Income sources are explained and shown
[ ] The sources are animated
[ ] Custom wanted subtitles
[ ] players can skip the animated scoreboard


Sanitizers do not work with JNI

Needs investigation.

Tried suppressions feature, it did not work for me yet.  Also tried LSAN_OPTIONS
Looking further

Most of leaks are reported in libjvm.so and originate from standard jni memory allocation routines; there is a report that jvm can be rebuilt in a way that the leaks are shown correctly.

We can avoid JNI by using cached Calcite responses. So far it is supported only in ArrowBasedExecuteTest. You can run it with --build-rel-alg-cache=<file> to collect Calcite responses and then run it again with --use-rel-alg-cache=<file> to avoid Calcite construction and any JNI calls. I tried it with ASAN some time ago and it worked fine. If it works for you, I can support this feature for all other test suites.

I have tried both options, total Calcite parsing time decreased from 28263 ms to 3753 ms; seems working

ASAN aborted after the first testcase of ArrowBasedExecuteTest, the following errors are not related to Calcite as expected. I believe one may continue ASAN enabling after this is fixed.
=================================================================
==377441==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x614000428440 in thread T0:
  object passed to delete has wrong type:
  size of the allocated type:   408 bytes;
  size of the deallocated type: 304 bytes.
    #0 0x7ff4c4b19c65 in operator delete(void*, unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:177
    #1 0x562c06d4e764 in query_template(llvm::Module*, unsigned long, bool, bool, QueryMemoryDescriptor const&, ExecutorDeviceType, bool, GpuSharedMemoryContext const&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x2034764)
    #2 0x562c06cef531 in Executor::compileWorkUnit(std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, RelAlgExecutionUnit const&, CompilationOptions const&, ExecutionOptions const&, GpuMgr const*, bool, std::shared_ptr<RowSetMemoryOwner>, unsigned long, signed char, bool, DataProvider*, std::unordered_map<int, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > > > > >&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1fd5531)
    #3 0x562c075d105d in QueryCompilationDescriptor::compile(unsigned long, signed char, bool, RelAlgExecutionUnit const&, std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, ColumnFetcher const&, CompilationOptions const&, ExecutionOptions const&, Executor*) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x28b705d)
    #4 0x562c069aa4e3 in Executor::executeWorkUnitImpl(unsigned long&, bool, bool, std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, RelAlgExecutionUnit const&, CompilationOptions const&, ExecutionOptions const&, std::shared_ptr<RowSetMemoryOwner>, bool, DataProvider*, std::unordered_map<int, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > > > > >&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1c904e3)
    #5 0x562c069afca2 in Executor::executeWorkUnit(unsigned long&, bool, std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, RelAlgExecutionUnit const&, CompilationOptions const&, ExecutionOptions const&, bool, DataProvider*, std::unordered_map<int, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > > > > >&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1c95ca2)
    #6 0x562c06f75ce8 in ExecutionResult RelAlgExecutor::executeWorkUnit(RelAlgExecutor::WorkUnit const&, std::vector<TargetMetaInfo, std::allocator<TargetMetaInfo> > const&, bool, CompilationOptions const&, ExecutionOptions const&, long, std::optional<unsigned long>)::{lambda(auto:1, bool, bool)#1}::operator()<unsigned long>(unsigned long, bool, bool) const (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x225bce8)
    #7 0x562c06fdb9dd in RelAlgExecutor::executeWorkUnit(RelAlgExecutor::WorkUnit const&, std::vector<TargetMetaInfo, std::allocator<TargetMetaInfo> > const&, bool, CompilationOptions const&, ExecutionOptions const&, long, std::optional<unsigned long>) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22c19dd)
    #8 0x562c06fddf5f in RelAlgExecutor::executeCompound(RelCompound const*, CompilationOptions const&, ExecutionOptions const&, long) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22c3f5f)
    #9 0x562c06fe2f5d in RelAlgExecutor::executeRelAlgStep(RaExecutionSequence const&, unsigned long, CompilationOptions const&, ExecutionOptions const&, long) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22c8f5d)
    #10 0x562c06fe5998 in RelAlgExecutor::executeRelAlgSeq(RaExecutionSequence const&, CompilationOptions const&, ExecutionOptions const&, long, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22cb998)
    #11 0x562c06fe875e in RelAlgExecutor::executeRelAlgQueryNoRetry(CompilationOptions const&, ExecutionOptions const&, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22ce75e)
    #12 0x562c06feb15d in RelAlgExecutor::executeRelAlgQuery(CompilationOptions const&, ExecutionOptions const&, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22d115d)
    #13 0x562c0660f6b0 in TestHelpers::ArrowSQLRunner::(anonymous namespace)::ArrowSQLRunnerImpl::runSqlQuery(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, CompilationOptions const&, ExecutionOptions const&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18f56b0)
    #14 0x562c06610a69 in TestHelpers::ArrowSQLRunner::(anonymous namespace)::ArrowSQLRunnerImpl::run_multiple_agg(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ExecutorDeviceType, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18f6a69)
    #15 0x562c0662154e in TestHelpers::ArrowSQLRunner::run_simple_agg(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ExecutorDeviceType, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x190754e)
    #16 0x562c05e7db6a in Distributed50_FailOver_Test::TestBody() (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1163b6a)
    #17 0x562c06608217 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18ee217)
    #18 0x562c065cfab4 in testing::Test::Run() [clone .part.0] (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18b5ab4)
    #19 0x562c065d071c in testing::TestInfo::Run() [clone .part.0] (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18b671c)
    #20 0x562c065d0e99 in testing::TestSuite::Run() [clone .part.0] (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18b6e99)
    #21 0x562c065d46d1 in testing::internal::UnitTestImpl::RunAllTests() (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18ba6d1)
    #22 0x562c065d5342 in testing::UnitTest::Run() (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18bb342)
    #23 0x562c05c9aa87 in main (/omniscidb/build/Tests/ArrowBasedExecuteTest+0xf80a87)
    #24 0x7ff4c07df082 in __libc_start_main ../csu/libc-start.c:308
    #25 0x562c05db7b2d in _start (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x109db2d)

0x614000428440 is located 0 bytes inside of 408-byte region [0x614000428440,0x6140004285d8)
allocated by thread T0 here:
    #0 0x7ff4c4b18587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
    #1 0x562c06d4e993 in query_template(llvm::Module*, unsigned long, bool, bool, QueryMemoryDescriptor const&, ExecutorDeviceType, bool, GpuSharedMemoryContext const&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x2034993)
    #2 0x562c06cef531 in Executor::compileWorkUnit(std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, RelAlgExecutionUnit const&, CompilationOptions const&, ExecutionOptions const&, GpuMgr const*, bool, std::shared_ptr<RowSetMemoryOwner>, unsigned long, signed char, bool, DataProvider*, std::unordered_map<int, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > > > > >&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1fd5531)
    #3 0x562c075d105d in QueryCompilationDescriptor::compile(unsigned long, signed char, bool, RelAlgExecutionUnit const&, std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, ColumnFetcher const&, CompilationOptions const&, ExecutionOptions const&, Executor*) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x28b705d)
    #4 0x562c069aa4e3 in Executor::executeWorkUnitImpl(unsigned long&, bool, bool, std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, RelAlgExecutionUnit const&, CompilationOptions const&, ExecutionOptions const&, std::shared_ptr<RowSetMemoryOwner>, bool, DataProvider*, std::unordered_map<int, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > > > > >&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1c904e3)
    #5 0x562c069afca2 in Executor::executeWorkUnit(unsigned long&, bool, std::vector<InputTableInfo, std::allocator<InputTableInfo> > const&, RelAlgExecutionUnit const&, CompilationOptions const&, ExecutionOptions const&, bool, DataProvider*, std::unordered_map<int, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::unordered_map<int, std::shared_ptr<ColumnarResults const>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<ColumnarResults const> > > > > > >&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1c95ca2)
    #6 0x562c06f75ce8 in ExecutionResult RelAlgExecutor::executeWorkUnit(RelAlgExecutor::WorkUnit const&, std::vector<TargetMetaInfo, std::allocator<TargetMetaInfo> > const&, bool, CompilationOptions const&, ExecutionOptions const&, long, std::optional<unsigned long>)::{lambda(auto:1, bool, bool)#1}::operator()<unsigned long>(unsigned long, bool, bool) const (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x225bce8)
    #7 0x562c06fdb9dd in RelAlgExecutor::executeWorkUnit(RelAlgExecutor::WorkUnit const&, std::vector<TargetMetaInfo, std::allocator<TargetMetaInfo> > const&, bool, CompilationOptions const&, ExecutionOptions const&, long, std::optional<unsigned long>) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22c19dd)
    #8 0x562c06fddf5f in RelAlgExecutor::executeCompound(RelCompound const*, CompilationOptions const&, ExecutionOptions const&, long) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22c3f5f)
    #9 0x562c06fe2f5d in RelAlgExecutor::executeRelAlgStep(RaExecutionSequence const&, unsigned long, CompilationOptions const&, ExecutionOptions const&, long) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22c8f5d)
    #10 0x562c06fe5998 in RelAlgExecutor::executeRelAlgSeq(RaExecutionSequence const&, CompilationOptions const&, ExecutionOptions const&, long, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22cb998)
    #11 0x562c06fe875e in RelAlgExecutor::executeRelAlgQueryNoRetry(CompilationOptions const&, ExecutionOptions const&, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22ce75e)
    #12 0x562c06feb15d in RelAlgExecutor::executeRelAlgQuery(CompilationOptions const&, ExecutionOptions const&, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x22d115d)
    #13 0x562c0660f6b0 in TestHelpers::ArrowSQLRunner::(anonymous namespace)::ArrowSQLRunnerImpl::runSqlQuery(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, CompilationOptions const&, ExecutionOptions const&) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18f56b0)
    #14 0x562c06610a69 in TestHelpers::ArrowSQLRunner::(anonymous namespace)::ArrowSQLRunnerImpl::run_multiple_agg(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ExecutorDeviceType, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18f6a69)
    #15 0x562c0662154e in TestHelpers::ArrowSQLRunner::run_simple_agg(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ExecutorDeviceType, bool) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x190754e)
    #16 0x562c05e7db6a in Distributed50_FailOver_Test::TestBody() (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x1163b6a)
    #17 0x562c06608217 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18ee217)
    #18 0x562c065cfab4 in testing::Test::Run() [clone .part.0] (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18b5ab4)
    #19 0x562c065d071c in testing::TestInfo::Run() [clone .part.0] (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18b671c)
    #20 0x562c065d0e99 in testing::TestSuite::Run() [clone .part.0] (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18b6e99)
    #21 0x562c065d46d1 in testing::internal::UnitTestImpl::RunAllTests() (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18ba6d1)
    #22 0x562c065d5342 in testing::UnitTest::Run() (/omniscidb/build/Tests/ArrowBasedExecuteTest+0x18bb342)
    #23 0x562c05c9aa87 in main (/omniscidb/build/Tests/ArrowBasedExecuteTest+0xf80a87)
    #24 0x7ff4c07df082 in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: new-delete-type-mismatch ../../../../src/libsanitizer/asan/asan_new_delete.cc:177 in operator delete(void*, unsigned long)
==377441==HINT: if you don't care about these errors you may set ASAN_OPTIONS=new_delete_type_mismatch=0
==377441==ABORTING

How to reproduce:
git clone https://github.com/intel-ai/omniscidb.git
python3 scripts/conda/make-m2-proxy.py # create proxy if needed

cmake -B /omniscidb/build -S /omniscidb/ -DENABLE_TESTS=on -DENABLE_CUDA=off -DENABLE_ASAN=off # build
cmake --build /omniscidb/build/ --parallel 4

cd build/Tests # build cache
./ArrowBasedExecuteTest --build-rel-alg-cache=cache.txt

cd ../.. # rebuild with ASAN
cmake -B /omniscidb/build -S /omniscidb/ -DENABLE_TESTS=on -DENABLE_CUDA=off -DENABLE_ASAN=on
cmake --build /omniscidb/build/ --parallel 16

cd build/Tests # run the test with cache and ASAN
./ArrowBasedExecuteTest --use-rel-alg-cache=cache.txt >asan.log

I've rerun the tests with ASAN_OPTIONS="detect_leaks=0" and ArrowBasedExecuteTest excluded. A new memory problem was found in UdfTest.
$ ./UdfTest
[==========] Running 7 tests from 1 test suite.
[----------] Global test environment set-up.
==2209597==ERROR: AddressSanitizer: requested allocation size 0x7fe144de3169 (0x7fe144de4170 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
#0 0x7fe14e3bc587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
#1 0x55c9ac652694 in (anonymous namespace)::get_clang_path(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) (/omniscidb/build/Tests/UdfTest+0xce1694)
#2 0x55c9ac6533e6 in UdfCompiler::UdfCompiler(CudaMgr_Namespace::NvidiaDeviceArch, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >) (/omniscidb/build/Tests/UdfTest+0xce23e6)
#3 0x55c9ac543792 in (anonymous namespace)::SQLTestEnv::SetUp() (/omniscidb/build/Tests/UdfTest+0xbd2792)
#4 0x55c9ac6117ea in testing::internal::UnitTestImpl::RunAllTests() (/omniscidb/build/Tests/UdfTest+0xca07ea)
#5 0x55c9ac614132 in testing::UnitTest::Run() (/omniscidb/build/Tests/UdfTest+0xca3132)
#6 0x55c9ac48ae7c in main (/omniscidb/build/Tests/UdfTest+0xb19e7c)
#7 0x7fe1448cd082 in __libc_start_main ../csu/libc-start.c:308
==2209597==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: allocation-size-too-big ../../../../src/libsanitizer/asan/asan_new_delete.cc:104 in operator new(unsigned long)
==2209597==ABORTING
The leak detector does not work for most tests which invoke Calcite via JNI and create unreadable log.

The error in UdfTest can be mitigated via adding /usr/lib/llvm-12/bin to PATH; the problem does not happen on the test main execution branch. The change happens due to transition to LLVM 12.

Here is a generic test run log
asan.log

Ok, thanks! I have an idea of where the first issue is coming from, but fixing it opened up some additional issues. Will look at the UDF test issue afterwards.

TSAN run results
when a test fails due to race condition, tsan hangs, see StringDictionaryTest, ParallelSortTest
Here is the log
tsan.log

ASAN should be fixed w/ https://github.com/intel-ai/omniscidb/commit/676b290418968a1d4a45b5d54cd5666718ff849e

@alexbaden thanks, now all test cases of ArrowBasedExecuteTest are able to run; there is still a problem with TBB:
=================================================================
==485705==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 24600 byte(s) in 3 object(s) allocated from:
    #0 0x7f50ff502787 in operator new[](unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:107
    #1 0x7f50fc9c966f  (/lib/x86_64-linux-gnu/libtbb.so.2+0x2766f)

SUMMARY: AddressSanitizer: 24600 byte(s) leaked in 3 allocation(s).

The guys from TBB blame pthread_key_create() https://community.intel.com/t5/Intel-oneAPI-Threading-Building/TBB-memory-leaks-even-when-entire-program-is-commented-out/m-p/1009653

@aregm said tbb had improved, so maybe we need to update. We could also add a suppression: https://github.com/intel-ai/omniscidb/blob/jit-engine/config/asan.suppressions for TSAN I would prefer not to do that, but for ASAN it's probably ok.

With exception to TBB and JNI it seems there are no ASAN issues in most tests. The following tests have JNI issues, @ienkovich maybe Ilya's option is needed here:
CorrelatedSubqueryTest
DataRecyclerTest
ArrayTest
TableFunctionsTest
NoCatalogSqlTest
SQLHintTest
TopKTest
ResultSetArrowConversion
ArrowStorageSqlTest
StringFunctionsTest
UdfTest
ResultSetTest
GroupByTest
JoinHashTableTest
BumpAllocatorTest

Let's split ASAN into two remaining tasks:

TBB -- can the issues be rectified by either building TBB from source with ASAN enabled, or by adding a suppression?
Let's ignore the remaining tests in favor of getting ArrowBasedExecuteTest working, and go from there. Likely the remaining tests have JNI issues.

I'm ok with both solutions;
I will try to setup CI for ArrowBasedExecuteTest separately

I've heard there are plans to merge hdk and omniscidb workspaces; better to do 2 after workspaces are merged

There are plans to do that, but I don't know that it will happen in the next few weeks.

Here is a PR which is required to add tests to HDK build, https://github.com/intel-ai/omniscidb/pull/581

When running tests in HDK, I get multiple problems, see the log below. Some of them are related to function get_root_abs_path
LastTest.log

The following patch seems to fix the issue
diff --git a/OSDependent/Unix/omnisci_path.cpp b/OSDependent/Unix/omnisci_path.cpp
index 0a113fe53..40f98729e 100644
--- a/OSDependent/Unix/omnisci_path.cpp
+++ b/OSDependent/Unix/omnisci_path.cpp
@@ -52,9 +52,10 @@ std::string get_root_abs_path() {
     /* Despite the dlinfo man page claim that l_name is absolute path,
        it is so only when the location path to the library is absolute,
        say, as specified in LD_LIBRARY_PATH. */
+
     std::filesystem::path abs_exe_dir(std::filesystem::absolute(
         std::filesystem::canonical(std::string(link_map->l_name))));
-    const auto mapd_root = abs_exe_dir.parent_path().parent_path();
+    const auto mapd_root = abs_exe_dir.parent_path().parent_path().parent_path();
     return mapd_root.string();
   }
 #endif

The next problem is at
  void createCalciteServerHandler(JNIEnv* env, const std::string& udf_filename) {
    jclass handler_cls = findClass(env, "com/mapd/parser/server/CalciteServerHandler");

I need to correct the classpath somehow

The problem with paths is moved to https://github.com/intel-ai/hdk/issues/120

It was good @Garra1980 suggested to try launching ASAN with omniscidb. Another problem manifested itself - Github virtual machines do not have enough virtual memory for ASAN to start due to their virtualization settings.
==12699==ReserveShadowMemoryRange failed while trying to map 0xdfff0001000 bytes. Perhaps you're using ulimit -v

I have tried a small test and it worked; is it possible to scale a data size via option for ArrowBasedExecuteTest?
https://github.com/leshikus/ghtest/actions/runs/3585577524/jobs/6033734506


Integration with almond vs require

This really isn't an issue, but I wanted to post here to see if I could get some help.  I really need to integrate js-xlsx into our project.  The issue that I'm running into is that we are using almond due to the fact that we are using cesium.  Our underlying project, doesn't use AMD so we really haven't had any conflicts with this issue so far.  Since almond(https://github.com/requirejs/almond), a lightweight version of require, the lib defines 'define' a little differently.  I'm having issues getting jszip and js-xlxs to load properly.  I'm trying to work through the issues with cesium to use the non almond version, but that is a pretty big change.  So far, I haven't had any luck.  I did find the amdclean(https://github.com/gfranko/amdclean) project as well which turns AMD code into pure Javascript.  If anyone has any suggestions, or there's a way to build this library in away that will load in almond, please let me know.
TIA,
Scott

@scottnc27603 please check against the latest version.  We included a require.js demo


adds a fiat-to-sat and sat-to-fiat calculator to the bot

adds a fiat-to-sat and sat-to-fiat calculator to the bot
let me know what you thing @AE9999

need some time and fixing to get this to work haha

Thanks. I will take a look when I have time (somewhere this weekend).

Thanks. I will take a look when I have time (somewhere this weekend).

I mean it works, I just needed to edit and adapt multiple times to make it work. Thatswhy so many commits.
You can also have a look at my fork, the 3 files mod.rs, business_logic.rs an commands,rs can be just taken, no other stuff in there except your code and the new function. I removed the AS connection stuff, will look at this another time.

okay sorry I am a git beginner, should I close this PR then? And squash those commits? or will this PR be adapted then?

Some quick git tips. You can actually pretty easy squash commits and override existing branches (as long as there are no merge commits). In your case one should do (in bash):
git rebase --interactive 5cbf0d3
Pick the first (top) commit and change all other 'picks' to 'f' (fold)
Next override the existing branch using:
git push origin master -f

okay sorry I am a git beginner, should I close this PR then? And squash those commits? or will this PR be adapted then?

Some quick git tips. You can actually pretty easy squash commits and override existing branches (as long as there are no merge commits). In your case one should do (in bash):
git rebase --interactive 5cbf0d3
Pick the first (top) commit and change all other 'picks' to 'f' (fold)
Next override the existing branch using:
git push origin master -f

No need to close the request, just perform it and and the mr should update automatically. (just make sure you always keep a local copy of your code in another directory).

I dont know what happened, I did what you said but still there a a lot of commits. and on shell there are no more commits to fold.
Maybe ill just delete my fork and restart a fresh one. Ive save all the code changes locally

okay cool, you can check my description for new ideas and also if you like adapt your description which makes installing th bot easier for newbs a bit.
I recognized a crash of the container this night, will check on this if it happends again. Had no time to check logs

@AE9999 thanks for merging, checkout your description and readd your personal details, because my README.md ist different then yours, added my node and my matrixroom
I am testing the next project on a branch (send2lnadress) and it works without no problems, will make a PR for you today if its okay?

Hi @warioishere I will restore the read.me. Thanks for any future PR I will look at them, but I can't promise it will be before next Saturday / Sunday.

Hi @warioishere I will restore the read.me. Thanks for any future PR I will look at them, but I can't promise it will be before next Saturday / Sunday.

not much more to add from my side. I added some things that could be useful, and I think the next step, making your bot work as an AS Service is too much for my skills. I would suggest you to try to work on this one next, because it adds a lot to the privacy and security of this project.
Maybe you can check, how the matrix telegram bridge bot is integrated as a Application Service:
https://github.com/mautrix/telegram
further details on how configuration is done:
https://docs.mau.fi/bridges/python/setup.html?bridge=telegram
also available as docker:
https://docs.mau.fi/bridges/general/docker-setup.html?bridge=telegram
the thing is, the AS Service is a own client that needs a connection that the matrix homeserver can talk to.
It will then create a real bot that cannot be spied or hacked.
thats also the way on how to configure on which instances the bot can be used (all matrixservers or just specific ones.
I think this needs a rewrite and reorganisation of your code and how it works currently, and that is defintily too much for me.
Maybe you can check on this project if you ever have time. I'll continue to contribute small things if I see things to optimize :)

Thanks that is precisely the direction I want to go, should have an update in a week or so.

Thanks that is precisely the direction I want to go, should have an update in a week or so.

sorry the links are not so useful, its a python bases bridge, I just wanted to show an example, maybe the config files of this may help you more:
https://github.com/mautrix/telegram/blob/master/mautrix_telegram/example-config.yaml
then they mostly have some tools that turn the configuration file of the Service into a registration yaml that needs to be put on the directory of your homeserver and needs to be added like this in the homeserver.yaml:
app_service_config_files:
  - /etc/matrix-synapse/double_puppet.yaml
  - /etc/matrix-synapse/mautrix-telegram-bot.yaml

and here general instructions from matrix:
https://www.matrix.org/blog/2015/03/02/introduction-to-application-services/
Hope I gave you a small idea!
greetings!


Files UWP

Thank you for reporting, what were you trying to do when this happened?

@SeI7H What is the version number that you are using? You should be able to check it by going to settings > about.

@SeI7H Please let us know what needs to be done to reproduce this bug. Did you interact with the PathBar before this occurred?
Otherwise, I'll close this bug soon.


Can you release training log for ViT 800 epochs?

As the title implies, can you release both pretraining and fine-tuning log?

As the title implies, can you release both pretraining and fine-tuning logs for ViT-800?

+1. The best result I can reproduce for VIT is 82.8. I use the provided config file.

+1. The best result I can reproduce for VIT is 82.8. I use the provided config file.


Bad practice

https://github.com/w96k/sitemap-cli-tools/blob/727953c87133956fa3e274c0c7662dea4baebd8e/web/routes/api.js#L80
think about it.

Feel free to send PR (hue hue)


⚠ Bofrost.be has degraded performance

In 8f834b2, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 2499 ms

Resolved: Bofrost.be performance has improved in e8be1ac after 8 minutes.


Issue with multiple record list filter of multi value DateTime field

DateTime field value didn't correctly format in the query for IN the operator. (~psql)

Working on fix for single value dateTime along with multivalue dateTime field.

Ref branch# 2022.9.x-fix-multi-date-filter

The query issue is resolved, but result set is still empty looks like parsing of the result is failing for multi date with ISO format.

The whole logic for filtering multi-value fields might need to be re-visited; specifically, a bit of the generated query is JSON_UNQUOTE(JSON_EXTRACT(compose_record.values, '$.dt[0]')) which would again work only over the first element (MySQL dialect output).
What we'll probably need to do is iterate the JSON, cast all values, and then query over that.

for today: assist with this task


Repo sync for protected CLA branch

The pull request is created from master637733955080349401 to master to fix git push error for protected CLA branch

@ghogen : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Access NAS server

Hi,
Are there any plans to add support to retrieve photos from a local NAS server?   I've both a OSNAP and ASUSTOR NAS connected to my network and would like to be able to access the photos stored in these 2 NAS servers.
I'm currently using both Fire TV stick and Xiaomi TV Box S (2nd Gen) hocked up to my TVs in the bedroom and living room.  AVES Gallery are installed in both of these tv boxes.
Thank you.
SH Loo

Related:

#126
#709

I'd prefer this be handled by allowing the user to manually specify "media folders" - e.g. just let the app have the file browser function, then the user can navigate to and select whatever additional directories they may require (and de-select any they wish as well). This would ensure compatibility across any type of storage that may be mounted without requiring the additional development overhead for this app to maintain support for any specific protocol.
On my device, for example, I have my local android storage, but also have 3 SMB mounts, 1 WebDAV, 1 NFS, and my Nextcloud account; with android now having unified storage (and with low latency / high bandwidth WiFi finally becoming more normalized, ensuring the feature's actually useful lol), seems like a (hopefully) simpler way to go about it than managing various providers/protocols within the app.

May as an Idea.
A minimal server application that collect all Tags and preview in a certain directory and sub-directory. (may with inotify) and share it over rest with AVES may a small solution.
Or work with an already existing Photoserver like immich.