Weird behaviour when using between multiple projects

Describe the bug
Hi, I have a really weird bug happening in my project.
I have two projects: ui_core and web. The ui_core is just a package that contains shared elements between the web project and the mobile one. The web is a flutter project for web that uses the ui_core.
The ui_core project has slang implemented and I created a custom extension to access the slang:
import 'package:flutter/widgets.dart';
import 'translations_core.g.dart';

extension AppLocalizationsCoreX on BuildContext {
  TranslationsCoreEn get i18nCore =>
      LocaleSettings.instance.currentTranslations;
}

This is exported from this package and used in the web.
The web has also slang implemented. I wrap my web application using the TranslationProvider generated in the web project, and also use the LocaleSettings.setLocaleRaw to change language using the one provided by the generated file in the web project.
Everything seems working fine, except for one page
In the web I have this page with these elements:

class OnboardingWeb extends StatelessWidget {
  const OnboardingWeb({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Row(
      children: [
        _Item(),
        Expanded(
          flex: 55,
          child: LoginView(
            onGoogleSignIn: () => signingWithGoogle(context),
            onTapPrivacy: () =>
                context.beamToReplacementNamed(AppRoutes.privacy),
            languageSelectorWeb: const LanguageSelector(),
          ),
        ),
      ],
    );
  }

  Future<void> signingWithGoogle(BuildContext context) async {
    ....
  }
}

class _Item extends StatelessWidget {
  const _Item({super.key});

  @override
  Widget build(BuildContext context) {
    return _BenefitCarouselItem(
      text: context.i18nCore.auth.effectMessage,
    );
  }
}

class _BenefitCarouselItem extends StatelessWidget {
  const _BenefitCarouselItem({
    Key? key,
    required this.text,
  }) : super(key: key);

  final String text;

  @override
  Widget build(BuildContext context) {
    return Text(
      text,
      textAlign: TextAlign.center,
      style: context.textTheme.headlineSmall,
    );
  }
}

The text inside the _BenefitCarouselItem  isn't updated. But if I use context.i18nCore.auth.effectMessage instead of the text in the parameter, the text is updated. This is weird because it's the same text.
Can anyone know why that is happening? Because of this issue I'm afraid that the same problem is happening in other parts of the app that I can't see.

If I understood your correctly, then the widget might not be updated because the widget is not marked to be rebuilt.
The expression context.i18nCore.auth.effectMessage seem to be a bit misleading because it does not access the context at all (it may be syntactic sugar so it's ok).
What you need is final t = Translations.of(context) (the Translations class from web). This will mark the widget to be rebuilt when the locale changes. Alternatively, just Translations.of(context) is enough.
Usually, you only need this in the app settings page or other selected pages.
In summary, context.i18nCore.auth.effectMessage is a static call so Flutter does not know that this is locale dependent. Translations.of(context) is needed.

Thanks @Tienisto , changing the extension from the core package to the code below fixed the issue
extension AppLocalizationsX on BuildContext {
  TranslationsCoreEn get i18nCore => Translations.of(this);
}

Doing that I was asked to provide the TranslationProvider to my widget. Adding both provider from the web, and from the core, fixed the issue.
The weird thing is that the extension that I use in the web needs to be:
extension AppLocalizationsX on BuildContext {
  TranslationsEn get i18n => LocaleSettings.instance.currentTranslations;
}

Because if I change to Translations.of(this), it stops working. Do you know why? At least everything is working now

Because if I change to Translations.of(this), it stops working.

What exact error message do you get?

I don't get any errors, but when I change the language using the LocaleSettings.setLocaleRaw provided by the generated file in the web project, it doesn't update the texts from the web file (the core keeps working). Using LocaleSettings.instance.currentTranslations it updates the text.

It seems that having 2 different TranslationProvider doesn't work.
At least the following code
void main() {
  runApp(
    core.TranslationProvider(
      child: TranslationProvider(child: const MyApp()),
    ),
  );
}

Throws an error:
The following assertion was thrown while finalizing the widget tree:
Multiple widgets used the same GlobalKey.

The key [LabeledGlobalKey<_TranslationProviderState<BaseAppLocale<dynamic, dynamic>, BaseTranslations<dynamic, dynamic>>>#47a90] was used by multiple widgets. The parents of those widgets were:
- InheritedLocaleData<AppLocale, StringsEn>
- [root](renderObject: RenderView#2eea0)
A GlobalKey can only be specified on one widget at a time in the widget tree.

I will dive deeper if there is a workaround for this

It's weird, I'm doing exactly that:
import 'i18n/translations.g.dart' as web_translations;
import 'package:ui_core/i18n/translations_core.g.dart'  as core_translations;

...

appWidget = web_translations.TranslationProvider(
        child: core_translations.TranslationProvider(child: appWidget));
    runApp(_monitorPerfomanceWidget(appWidget));

And it's working. The only issue is that I need to use the LocaleSettings.instance.currentTranslations to get the web texts updated when changing translations and Translations.of(this) to the core ones.

Weird.
I added a fixed so that it works on my machine. Now it should support multiple TranslationProvider.
You could try it out:
dependency_overrides:
  slang_flutter:
    git:
      url: https://github.com/slang-i18n/slang.git
      path: slang_flutter
      ref: be312457379a547efe43c85cda5fa6873f4710f2

It should throw an error on your side, at least the minimal reproducable example showed it:
https://github.com/slang-i18n/slang/tree/test-multiple-providers
import 'package:core/gen/strings.g.dart' as core;
import 'package:flutter/material.dart';
import 'package:frontend/gen/strings.g.dart' as frontend;

void main() {
  runApp(
    core.TranslationProvider(
      child: frontend.TranslationProvider(child: const MyApp()),
    ),
  );
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      title: 'Flutter Demo',
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatelessWidget {
  const MyHomePage({super.key});

  @override
  Widget build(BuildContext context) {
    return const Scaffold(
      body: Center(
        child: Text('Hello'),
      ),
    );
  }
}

Thanks, I'm using the package from the git now.
I started to see the errors when started running the mobile project. I think the web one hided the logs so I didn't see, or maybe the error isn't happening in the web, because the translations worked there. But now I'm using the one from the github and it's working fine without the errors.

Fixed in 3.23.0


Parallelize model run upload scripts

I made some improvements to the upload scripts to make things faster, namely, using the multiprocessing model to do uploads in parallel. A new CLI argument, --parallelism, allows the user to specify the number of concurrent uploads to do at once. It defaults to the value returned by os.cpu_count(). This significantly speeds up upload when you're trying to get a large number of model runs loaded into your system quickly.
I also added the scripts/ subdirectory to the pre-commit config.

@BryonLewis to hopefully ease review, this commit is really where all the changes happen. The rest are formatting changes


Youtube: --all-subs doesn't download automatically generated transcripts: "video doesn't have subtitles"

Checklist

[x] I'm reporting a broken site support
[X] I've verified that I'm running youtube-dl version 2020.01.24
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones

Verbose log
youtube-dl -v --skip-download --all-subs https://youtu.be/i4Rr5wXSXII
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--skip-download', '--all-subs', 'https://youtu.be/i4Rr5wXSXII']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.01.24
[debug] Python version 3.8.1 (CPython) - Linux-5.4.19-x86_64-AMD_Ryzen_Threadripper_2990WX_32-Core_Processor-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[youtube] i4Rr5wXSXII: Downloading webpage
[youtube] i4Rr5wXSXII: Downloading video info webpage
WARNING: video doesn't have subtitles
[debug] Default format spec: bestvideo+bestaudio/best

Description
--all-subs doesn't download automatically generated YouTube transcripts. The above command doesn't produce any subtitles files.

@dstftw It should be complete now.


After installing Meteor <IP_ADDRESS> Console shows the persistent Session name and values

Hi There,
It seems after updating Meteor to <IP_ADDRESS> yesterday this package is giving the persistent Session names and values in the console. I have checked my code couple of times and felt I should share this with the community.
Please let me know if this is a cosmetic Bug or is it just me.
Thanks
Ajay

@nagarajay Hi Ajay! This was an issue with 0.4.2 where some debug statements were accidentally committed. They've been removed in 0.4.3 :)

Thanks a lot. Cheers!!


bump dependencies that CI fails upgrading

Test to check why CI fails.

Check out a preview of your pull request here: https://app-servicecanadalabs-dev-pr-preview-1.azurewebsites.net


Request to Increase Data Acquisition Rate for Hands

Hello,
Thank you for the great work done previously!
I encountered an issue while using this project. The data acquisition rate for both hands from Vision Pro is 30Hz, which is insufficient for precise control. Is it possible to increase the reading frequency?
Thank you very much.

@ark338 I have same questions. Do you solve it?

Update: I measured the stream frequency. It receives data at 60-70hz.


UDP unicast callback "MessageReceived" does not get invoked

// Socket Setup
m_exclusive_conn = new UdpSocketReceiver();
m_exclusive_conn.MessageReceived += OnReceiveData;
m_exclusive_conn.StartListeningAsync(m_port).Wait();
Program is able to receive Broadcast UDP messages, however unicast UDP messages appear to be excluded
What is required to support UDP broadcast and unicast messages on a socket ?

That setup should be fine for receiving unicast also. What are the characteristics of sender/receiver - platforms, physical/simulator, connection type?

HI Ryan,
The sender is a xamarin uwp application and receiver is a commercial controller, these devices are connected in the same wifi.


:id must be in the updateable_fields for RC2?

I'm working on converting an application over to the jsonapi_rc2 branch. I just noticed that a resource must have :id in its updateable_fields or else it receives a Param not allowed error (because :id is not in the formatted_allowed_fields). But, if you leave :id out of the data, you receive an "A key is required" error.
What is the intended behavior? I'm working on this today and can implement any changes that are required.

That behavior was not intended.  Since type and id are required fields in the spec I'd like to remove them from being processed by the updateable_fields and createable_fields functions. In other words the resource shouldn't be able to override the spec. I added a hack where I insert the id field in the inherited method. This was done to allow a formatter to be set for the id, but I've planning on revisiting this.

Closed by https://github.com/cerebris/jsonapi-resources/pull/79.

So what was the conclusion on this?  I've tried updating to the 0.7.1.beta1 and I can't either include the ID on the payload nor omit it for the reasons @barelyknown mentions.  Little confused, little help?


ESLint v9 migration

See:

Migrate to v9.x - ESLint - Pluggable JavaScript Linter
Flat config rollout plans - ESLint - Pluggable JavaScript Linter

This will be a major version update since it requires changes to the ESLint configuration file eslintrc and Node.js requirements in package.json

Migration will move forward once the public version of typescript-eslint v8, with full support for eslint v9, is released.


Issue 2265 solved

Add a new parameter which is not in the dictionary while having check_already_exists=True and error that throws an error if while having check_already_exists=False the user tries to change an existing parameter.
Fixes #2265

Hello @brosaplanella , I have raise a PULL request. Can you review it?


Change all tf.nn.dropout to tf.layers.dropout

This will eliminate a lot of the nastiness with having a pdrop_value AND a pkeep property on each class, and having to remember to do 1 - pdrop_value everwhere.  It will also make TF more consistent with the other frameworks.  Finally, it makes it possible to have the TRAIN_FLAG() be a boolean (or boolean placeholder) which is simpler and cleaner.

Tracking this here: https://github.com/dpressel/baseline/compare/feature/v1...feature/tf-layers-dropout?expand=1

PR: https://github.com/dpressel/baseline/pull/204

merged


[Bug] Bonemealing unripe apples does nothing.

Current behavior
Currently you can use bonemeal on unripe apples. However apples don't advance a stage when this is done.
Expected behavior
I'd expect to be able to use bonemeal at all, or to have it advance a growth stage.
Admittedly this is learned behaviour from other mods (primarily Pam's). However if using bonemeal on fruit is not a thing in this, then it shouldn't consume my bonemeal.
Screenshots

And in survival, you can see it decreases my bonemeal stack but doesn't affect the apple growth (sorry for the apple visibility being lacking, I hope you get the gist though).

Minecraft version 1.12.2 (Latest release)
Forge version <IP_ADDRESS>55 (Latest stable release)
Dynamic Trees version 1.12.2-0.7.6 (Latest release on CF)

Oh..   Thanks for reporting that. Somehow I never noticed it was doing that.  I'll have it fixed before the next alpha version

Fixed in dev.


Isolating the imdb train images to folders

I would like to train the model with imagenet and see if it can improve the performance. Can you please give me a script that can seperate the  imdb dataset to various folders with respect to the age?

@yu4u
I used the following code to segregate all twelve years old people into seperate folder.
import numpy as np
from utils import get_meta
from shutil import copy
import os

db = "imdb"
mat_path = "data/{}_crop/{}.mat".format(db, db)
print (mat_path)
full_path, dob, gender, photo_taken, face_score, second_face_score, age\
    = get_meta(mat_path, db)

print("#images: {}".format(len(face_score)))
print("#images with inf scores: {}".format(np.isinf(face_score).sum()))

img_paths = []
cols, rows = 4, 3
img_num = cols * rows
path_root = "data/{}_crop/".format(db)

for i in range(len(age)):
	if age[i] == 12:
		print (full_path[i][0])
		copy(path_root + "/" +str (full_path[i][0]), "sorted")

but still there are lots of false negatives inside the folder "sorted". am I doing the sorting in a right way?

I have a python program which can generate annotation file in the format of IMDB-WIKI dataset. Will it suit your need by generating a .mat file with only 12 years old picture?
My programme:
https://github.com/galoiscch/IMDB-WIKI-annotation-tool/blob/master/IMDB_WIKI_tool.py

This part is related to generating new .mat file
    def save_mat_file(self):         save_counter=0         full_pathList=np.zeros((len(self.annotationList),),dtype=np.object)         nameList=np.zeros((len(self.annotationList),),dtype=np.object)         for annotation_output in self.annotationList:             self.dobList.append(annotation_output['dob'])             self.photo_takenList.append(annotation_output['photo_taken'])             self.genderList.append(annotation_output['gender'])             nameList[save_counter]=annotation_output['name']             self.face_scoreList.append(annotation_output['face_score'])             self.second_face_scoreList.append(annotation_output['second_face_score'])             full_pathList[save_counter]=annotation_output['full_path']             save_counter+=1         photo={'dob':np.float64(self.dobList),'photo_taken':np.float64(self.photo_takenList),'full_path':full_pathList,'gender':np.float64(self.genderList),'name':nameList,'face_score':np.float64(self.face_scoreList),'second_face_score':np.float64(self.second_face_scoreList)}         path_to_mat_file=self.open_dir+"/"+"photo.mat" sio.savemat(path_to_mat_file, {'photo': photo})####################change the name of struct in there######################

    def save_mat_file(self):         save_counter=0         full_pathList=np.zeros((len(self.annotationList),),dtype=np.object)         nameList=np.zeros((len(self.annotationList),),dtype=np.object)         for annotation_output in self.annotationList:             self.dobList.append(annotation_output['dob'])             self.photo_takenList.append(annotation_output['photo_taken'])             self.genderList.append(annotation_output['gender'])             nameList[save_counter]=annotation_output['name']             self.face_scoreList.append(annotation_output['face_score'])             self.second_face_scoreList.append(annotation_output['second_face_score'])             full_pathList[save_counter]=annotation_output['full_path']             save_counter+=1         photo={'dob':np.float64(self.dobList),'photo_taken':np.float64(self.photo_takenList),'full_path':full_pathList,'gender':np.float64(self.genderList),'name':nameList,'face_score':np.float64(self.face_scoreList),'second_face_score':np.float64(self.second_face_scoreList)}         path_to_mat_file=self.open_dir+"/"+"photo.mat" sio.savemat(path_to_mat_file, {'photo': photo})####################change the name of struct in there######################

i want to organize imdb datasets to various folders based on the age,

for example creating a folder full of 12 old children

Let see if the programme solve your problem.
`
import numpy as np
from utils import get_meta
import cv2
import os
db = "photo"
#mat_path = "data/{}_crop/{}.mat".format(db, db)
mat_path="/home/milliontech/Desktop/li_pan_lung/photo.mat"###################.mat file path. Change to yours#######################
print (mat_path)
full_path, dob, gender, photo_taken, face_score, second_face_score, age
= get_meta(mat_path, db)
print("#images: {}".format(len(face_score)))
print("#images with inf scores: {}".format(np.isinf(face_score).sum()))
img_paths = []
#path_root = "data/{}_crop/".format(db)
photo_root="/home/milliontech/Desktop/li_pan_lung/"###################image directory root path. Change to yours#######################
new_root="/home/milliontech/Desktop/sort"###################new directory for saving sorted image. Change to yours#######################
full_path=[str(x) for x in full_path]
full_path = [eval(x) for x in full_path]
full_path = [x[0] for x in full_path]
full_path = [x.encode('ascii','ignore') for x in full_path]
for i in range(len(age)):
if age[i] == 32:
print (full_path[i])
img_cv = cv2.imread(os.path.join(photo_root,full_path[i]))
cv2.imshow("Gray image",img_cv)
cv2.waitKey(0)
new_path=os.path.basename(full_path[i])
cv2.imwrite(os.path.join(new_root,new_path),img_cv)
`
I used OpenCV to read the images and then write them out in a directory(which is created by me) named "sort".

okay thank you..let me try

I modified the code for the sake of windows.
import numpy as np
from utils import get_meta
import cv2
import os
from shutil import copy
db = "imdb"
mat_path = "data/{}_crop/{}.mat".format(db, db)
print (mat_path)
full_path, dob, gender, photo_taken, face_score, second_face_score, age\
    = get_meta(mat_path, db)

print("#images: {}".format(len(face_score)))
print("#images with inf scores: {}".format(np.isinf(face_score).sum()))

img_paths = []
photo_root = "data/{}_crop/".format(db)

for i in range(len(age)):
    if age[i] == 7:
        print (photo_root +str (full_path[i][0]))
        img_cv = cv2.imread(os.path.join(photo_root, full_path[i][0]))

        copy(photo_root + "/" + str(full_path[i][0]), "seven")

after copying the sorted file to the folder seven, i went through the pictures in that folder. it was not accurate. even the people who are old are present in this folder. may I know why?

@yu4u why is that the imdb cropped dataset is not proper in terms of age or am I segregating it in a wrong way?


Upgrade to ungoogled-chromium 87.0.4280.67-1

Ungoogled-chromium 87.0.4280.67-1 compiled and is running well on macOS. suppress-newer-clang-warning-flags.patch was removed since Chromium currently does not use any warning flags that Clang 11 does not support (at least, as far as I have observed).

Compiled successfully and is running well.
Thanks @Zoraver!


Cilium restarts if iptables lock is not held in 5s

The Reinitialize function runs some itpables commands. If any of these commands fails within 5 seconds, the Reinitialize returns an error causing this function to also return an error:
https://github.com/cilium/cilium/blob/ddd8fb1809e974ac7fa3a9f0b9f3dad26e0b2e73/daemon/daemon.go#L595-L599
However, the caller of the function fatals Cilium in case this happens:
https://github.com/cilium/cilium/blob/022349e1431ad67b5dc2f30d963b7f98ca38e017/pkg/datapath/ipcache/listener.go#L327-L335
IMO this should not happen since failing to acquire the iptables rules within 5 seconds might be sporadic and it can be successful if retried.

failing to acquire the iptables rules within 5 seconds might be sparodic

This is a fair statement if there's legitimate reason to believe there's a slow iptables interaction occurring in the environment. That said, 5s is a long time and if there is an application that's actually not releasing the xtables lock then Cilium could get stuck in this process. In that case per the comment in the second code snippet, the idea is to fail out to make the failure clear rather than silently continuing with an outdated ipcache map for an indeterminate time.


vm state documentation

Description
In the VM file source of GOCA, I found three LCM_STATE not documented:
// DiskResize lcm state
DiskResize LCMState = 62

// DiskResizePoweroff lcm state
DiskResizePoweroff LCMState = 63

// DiskResizeUndeployed lcm state
DiskResizeUndeployed LCMState = 64

Are they deprecated ?
They are just not documented here: vm states ONe 5.6
By the way, the link given to document the STATE and LCM_STATE values in the XSD file is dead.
The link is in this file: vm.xsd
FYI, it's this link:  http://docs.opennebula.org/stable/user/references/vm_states.html

Progress Status

[ ] Branch created
[ ] Code committed to development branch
[ ] Testing - QA
[ ] Documentation
[ ] Release notes - resolved issues, compatibility, known issues
[ ] Code committed to upstream release/hotfix branches
[ ] Documentation committed to upstream release/hotfix branches

Fixed here: https://github.com/OpenNebula/docs/pull/488, https://github.com/OpenNebula/one/pull/2894


Implement PDF Upload Functionality in TutorAI

Develop a feature in TutorAI that allows users to upload PDF versions of their course books or study materials. This functionality is crucial for enabling the application to analyze and extract content from these documents, which will be used to tailor the educational content, quizzes, and tests to the specific course material. The feature should support a range of PDF formats and sizes, ensuring a smooth and efficient upload process.
Acceptance Criteria:

[ ] File Upload Interface: Create a user-friendly interface for uploading PDF files.
[ ] File Type and Size Validation: Ensure the system accepts only PDF files and adheres to set size limits.
[ ] File Storage and Management: Implement secure and efficient storage solutions for the uploaded PDFs.
[ ] Content Extraction: Ability to accurately extract and interpret text from the uploaded PDFs.
[ ] Error Handling: Implement robust error handling for issues like file corruption or unsupported file formats.
[ ] User Feedback and Progress Indicators: Provide clear feedback and progress indicators during the file upload process.

To create the django file upload endpoint we can use the request.FILES
# views.py

from rest_framework.decorators import api_view
from rest_framework.response import Response
from rest_framework import status

@api_view(['POST'])
def upload_pdf(request):
    if 'pdf' in request.FILES:
        pdf_file = request.FILES['pdf']
        # Process or save pdf_file as needed
        # ...

        return Response({'message': 'PDF uploaded successfully!'}, status=status.HTTP_201_CREATED)
    else:
        return Response({'message': 'No PDF file found in request'}, status=status.HTTP_400_BAD_REQUEST)

# urls.py

from django.urls import path
from .views import upload_pdf

urlpatterns = [
    path('upload/', upload_pdf, name='upload_pdf'),
]

We might also need to parse the pdf into strings we can more easily work with. I found two dependencies that can help in that:
https://pypi.org/project/pypdf/ and https://pypi.org/project/pdfminer/

React typescript for making pdf upload:
// uploadService.ts

const uploadPDF = async (file: File): Promise<Response> => {
    const formData = new FormData();
    formData.append('pdf', file);

    const response = await fetch('http://yourdjangoapi.com/upload/', {
        method: 'POST',
        body: formData,
        // Add headers if necessary (e.g., for authentication)
    });

    return response;
};

export default uploadPDF;

// UploadComponent.tsx

import React, { useState } from 'react';
import uploadPDF from './uploadService';

const UploadComponent: React.FC = () => {
    const [selectedFile, setSelectedFile] = useState<File | null>(null);

    const handleFileChange = (event: React.ChangeEvent<HTMLInputElement>) => {
        if (event.target.files && event.target.files[0]) {
            setSelectedFile(event.target.files[0]);
        }
    };

    const handleSubmit = async (event: React.FormEvent) => {
        event.preventDefault();
        if (selectedFile) {
            try {
                const response = await uploadPDF(selectedFile);
                const data = await response.json();
                console.log(data);
                // Handle the response data
            } catch (error) {
                console.error('Error uploading file:', error);
            }
        }
    };

    return (
        <form onSubmit={handleSubmit}>
            <input type="file" accept="application/pdf" onChange={handleFileChange} />
            <button type="submit">Upload PDF</button>
        </form>
    );
};

export default UploadComponent;

It could be a sensitive topic to store user-uploaded pdfs

The current user interface shows the user the PDF.

I still have some problems with the backend receiving the pdf. I do not know why


feat: add zstd compression support

My team needed to add this to search over our log archives, upstreaming in case others can benefit.

Sorry, had to fix the Github workflows so that the tests run on your PR. Therefore I had to close and open it again.

Thanks for merging!


Error in example config

Example config in sample_rate var (https://github.com/Comcast/trickster/blob/master/cmd/trickster/conf/example.conf#L452) states that
sample_rate = 1

Can be used. However, if launched with that setting, trickster will crash due to failed conversion of int64 to go float.
Setting sample_rate to 1.0 allows trickster to launch.

Thanks for this report, I just committed a change to master for the main example.conf. We'll push a corresponding helm/kube/etc update to those projects with the next release. I'll keep this issue open until those changes are also made.

This change has been fully propagated and released. Thank you for the report and the associated PR!


[fix] cluster_name = cluster_name, not cluster_id

CATEGORY
Choose one

[X] Bug Fix
[ ] Enhancement (new features, refinement)
[ ] Refactor
[ ] Add tests
[ ] Build / Development Environment
[ ] Documentation

SUMMARY
Found these 3-year-old bugs while looking into some other stuff. I'm guessing this may have happened because of some copy paste errors. It probably isn't too important since they haven't been caught, but it's pretty irksome to look at, so here we go.
TEST PLAN
Passes tox.
REVIEWERS
@john-bodley

Codecov Report

Merging #8514 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8514   +/-   ##
=======================================
  Coverage   66.68%   66.68%
=======================================
  Files         449      449
  Lines       22684    22684
  Branches     2366     2366
=======================================
  Hits        15126    15126
  Misses       7420     7420
  Partials      138      138

Impacted Files
Coverage Δ

superset/connectors/druid/models.py
82.43% <ø> (ø)
:arrow_up:

superset/connectors/druid/views.py
69.12% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 571c474...aaa030f. Read the comment docs.


Cannot build demo quickstart because of error related to buildkit

Hi,
I'm unable to docker-compose the demo, I get the following error:
ERROR: Service 'warp-demo-wsgi' failed to build: the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled
I'm using a Ubuntu 20.04.
Tried to set env DOCKER_BUILDKIT=1 but I must be missing something.
Any advice? Thank you.

I think that you basically don't have buildkit installed. Please just update your docker version, here I found an instruction for you: https://docs.docker.com/engine/install/ubuntu/
Let me know if it still fails after that.

That was it! Thank you very much :)
BTW there's a little typo in your read me in the section "without docker - the old way"; you probably meant "flask run" and not "flash run", just guessing! :)
Thanks again!

You're right - it should be flask, I'm constantly doing this typo in my terminal too...
Thanks, and I hope you will find this software usefull.


[Cloud Sync] Merge remote-tracking branch 'origin/master' 2021-07-06

Summary
Merge master into cloud branch for cloud release
Release Note
NONE

cc @nickmisasi for PR review.


Do not update conda in CI builds

Need to check whether that fixes buildbot Docker-based builds.

[Current coverage][cc-pull] is 87.24%

Merging [#1957][cc-pull] into [master][cc-base-branch] will not change coverage

@@             master      #1957   diff @@
==========================================
  Files           299        299
  Lines         53344      53344
  Methods           0          0
  Messages          0          0
  Branches       5512       5512
==========================================
  Hits          46541      46541
  Misses         5883       5883
  Partials        920        920

Powered by Codecov. Last updated by [e890bbf...67813b0][cc-compare]
[cc-base-branch]: https://codecov.io/gh/numba/numba/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/numba/numba/compare/e890bbfa1c47fbdb43082ef5d98d6703ea5e3659...67813b0c67edb91bca4b65be262ad0d173425dd4
[cc-pull]: https://codecov.io/gh/numba/numba/pull/1957?src=pr

After @stefanseefeld's fixes to the Docker images, this passes fine.


fix heading level of Caffe docs

Currently both "Caffe Reference Model Support" and "Caffe Model Export Support" appearing in the references on the left.

LGTM!


Add XML formatting to DMN/BPMN editor outputs

This PR partially fixes #31.
There's still the issue with the ids on the BPMN editor, but diffs are now better than a one-liner.

jenkins retest this

@domhanak Thx for the review! I think the \ns will work on Windows, but it's worth the test :)


[IRGen] Add missing blocklist check for some CVW cases

rdar://127511568
The check was missing from getEnumTag and destructiveInjectEnumTag, which could cause the CVW functions to be used when the feature was blocked via blocklist.

@swift-ci smoke test

@swift-ci smoke test Linux


When there are 2 service links with same link name , only 1 of them is considered.

Server version - V0.25.0-rc1
Steps to reproduce the problem:
Create service S1.
Create service S2.
Create service S3.
Link S2 and S3 to service S1 with same link name - say "linkme"
Only one of the service link is programmed , usually the second service with the same service link is accepted.
Expected Behavior:
We should be able to link both S2 and S3 with the same link name.
If we cannot support this , API should error out with meaningful error message.

This is impossible with the current API (link name is the key of the map), and just a bad idea in the new API (array of service+name pairs).  Can we just not allow duplicate names?
Problems on both sides:

With duplicate names: Which one wins?  Both?  Which environment variables get set?
Without: But what if you provide an "as name" that's the same as a linked service's actual name?  Or you rename a service after the fact?

Now you can pass duplicated name for 2 diff services, so closing this bug.
@vincent99 if there are 2 duplicated name, both services are going to "win". The name would resolve to instances of both services. No cross-services-link name validation is done at this point - I'm letting user to control it. We can add validation later on per request.

Tested with server version - V0.27.0-rc3
I am able to have a service linked to 2 services with same link name . Both service links get saved.
DNS resolution on the link name , resolves to containers from both the services.


Add Romanian text

Rhymes don't produce the best random text. I'd prefer a prose text, if possible a contemporary one.


server: get default port optionally form environment var

If the default port 8080 is taken, you have to either change the app.run() or  use --port  on every invocation of the program. When trying out examples I don't want to do the first and forget the second.
This patch allows you set the environment variables LONA_DEFAULT_HOST and LONA_DEFAULT_PORT in your shell. These override the built-in defaults ('localhost' resp. 8080), and are in turn overridden by arguments to app.run() and startup commandline parameters.

had not realised that if you amend the title of the commit, that doesn't percolate through into the PR.
If you do
 LONA_DEFAULT_PORT=abc python example.py

you'll get a message:
value "abc" for environment variable "LONA_DEFAULT_PORT" cannot be converted to an integer port number

after which the system sys.exits with value 2 (same value as when using --port abc, however I realise you cannot have the above and specify --port 8081, as it will still error on the environment variable not converting. This might be circumvented by assigning the environment variable to port and doing the check-for-int-ness after setattr has optionally been updated by the commandline-parsing.
But if you have an incorrect LONA_DEFAULT_PORT setting, IMO that should be fixed, not suppressed by doing --port.

I did some testing and found out the check is just not necessary (sorry). Argparse does exactly what you would expect and will handle 8080 and "8080". So we just have to add port=env.get('LONA_DEFAULT_PORT', '8080') in these two spots, and everything should be fine.
https://github.com/lona-web-org/lona/blob/7f8ba4c28aa25453cf384707b80f33765316c744/lona/app.py#L432
https://github.com/lona-web-org/lona/blob/7f8ba4c28aa25453cf384707b80f33765316c744/lona/command_line/handle_command_line.py#L119

I did some testing and found out the check is just not necessary (sorry). Argparse does exactly what you would expect and will handle 8080 and "8080". So we just have to add port=env.get('LONA_DEFAULT_PORT', '8080') in these two spots, and everything should be fine.
https://github.com/lona-web-org/lona/blob/7f8ba4c28aa25453cf384707b80f33765316c744/lona/app.py#L432
https://github.com/lona-web-org/lona/blob/7f8ba4c28aa25453cf384707b80f33765316c744/lona/command_line/handle_command_line.py#L119

I did this but cannot re-apply for a review, probably because there is already a review request pending.

@AvdN: Yeah no worries, that's on me.
I think that's it! I will do some more local testing, but the patch should be mergeable in this state

Codecov Report
Patch coverage has no change and project coverage change: -0.02% :warning:

Comparison is base (7f8ba4c) 71.81% compared to head (113cfd8) 71.80%.

:exclamation: Current head 113cfd8 differs from pull request most recent head d55af0e. Consider uploading reports for the commit d55af0e to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #464      +/-   ##
==========================================
- Coverage   71.81%   71.80%   -0.02%
==========================================
  Files          84       84
  Lines        5745     5746       +1
  Branches     1235     1235
==========================================
  Hits         4126     4126
- Misses       1339     1340       +1
  Partials      280      280

Files Changed
Coverage Δ

lona/app.py
58.75% <ø> (ø)

lona/command_line/handle_command_line.py
0.00% <0.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@AvdN: Works like a charm! I added a small fix to your patch because LONA_DEFAULT_HOST did only work for apps, not for projects. Let me know if you are ok with this change, then we can merge.

I am not sure if I am supposed to be able to see the changes on github, at least I cannot find them. But go ahead, I'll see them at the latest when they are merged.

@AvdN: Thanks for contributing! :)


Add border to navigation links

Styles navigation links.
Fixes #54

Codecov Report

Merging #60 (b0841a2) into master (41cbafc) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #60   +/-   ##
=======================================
  Coverage   16.85%   16.85%
=======================================
  Files          36       36
  Lines         362      362
  Branches       72       72
=======================================
  Hits           61       61
  Misses        275      275
  Partials       26       26

Impacted Files
Coverage Δ

src/components/main/About.js
100.00% <ø> (ø)

src/components/main/categories/Categories.js
8.00% <ø> (ø)

src/components/main/history/History.js
0.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41cbafc...b0841a2. Read the comment docs.


Issues with --enable-benchmarks, nc_perf

While investigating #2626, I discovered a few issues with the performance tests/benchmarks.

Benchmark and performance tests can't be enabled under cmake.
make check fails to compile when --enable-benchmarks is specified w/out using parallel compilation.

Todo

[ ] Wire in benchmarks option to CMakeLists.txt
[ ] Make performance tests which require mpi conditional on parallel compilation.
[ ] Ensure performance checks pass.


Function returned an invalid response

Framework version: 1.0
Implementations: Spring Boot

Scenario

I created the archetype using

 mvn archetype:generate -DgroupId=my.service -DartifactId=my-service -Dversion=1.0-SNAPSHOT        -DarchetypeGroupId=com.amazonaws.serverless.archetypes  \
      -DarchetypeArtifactId=aws-serverless-springboot-archetype \
      -DarchetypeVersion=1.0.1

I run mvn package
Then started sam using sam local start-api --template sam.yaml with output

2018/03/11 15:12:19 Connected to Docker 1.35
2018/03/11 15:12:19 Fetching lambci/lambda:java8 image for java8 runtime...
java8: Pulling from lambci/lambda
Digest: sha256:c2ec8db145bbe44154dbddcdd2976ab8afd60d58d7d633dc98f53f657c0ef909
Status: Image is up to date for lambci/lambda:java8

Mounting my.service.StreamLambdaHandler::handleRequest (java8) at http://<IP_ADDRESS>:3000/{proxy+} [OPTIONS GET HEAD POST PUT DELETE PATCH]

You can now browse to the above endpoints to invoke your functions.
You do not need to restart/reload SAM CLI while working on your functions,
changes will be reflected instantly/automatically. You only need to restart
SAM CLI if you update your AWS SAM template.

Then tried to call the /ping endpoint at curl -s http://<IP_ADDRESS>:3000/ping | python -m json.tool

Expected behavior
The server replies with a "pong" message
Actual behavior
2018/03/11 15:12:46 Invoking my.service.StreamLambdaHandler::handleRequest (java8)
2018/03/11 15:12:46 Decompressing G:\Source\lambdas\my-service\target\my-service-1.0-SNAPSHOT.jar
2018/03/11 15:13:49 Mounting /c/Users/Daedin/AppData/Local/Temp/aws-sam-local-1520777566162251000 as /var/task:ro inside runtime container
START RequestId: 48dfc4ef-52c7-4674-8bf3-1853e7e8f4d4 Version: $LATEST
END RequestId: 48dfc4ef-52c7-4674-8bf3-1853e7e8f4d4
REPORT RequestId: 48dfc4ef-52c7-4674-8bf3-1853e7e8f4d4  Duration: 9083.56 ms    Billed Duration: 9100 ms        Memory Size: 512 MB     Max Memory Used: 15 MB
2018/03/11 15:14:01 Function returned an invalid response (must include one of: body, headers or statusCode in the response object): invalid character '-' after top-level value

which results in an Internal Server Error
Steps to reproduce
See description
Full log output
$ sam local start-api --template sam.yaml
2018/03/11 15:12:19 Connected to Docker 1.35
2018/03/11 15:12:19 Fetching lambci/lambda:java8 image for java8 runtime...
java8: Pulling from lambci/lambda
Digest: sha256:c2ec8db145bbe44154dbddcdd2976ab8afd60d58d7d633dc98f53f657c0ef909
Status: Image is up to date for lambci/lambda:java8

Mounting my.service.StreamLambdaHandler::handleRequest (java8) at http://<IP_ADDRESS>:3000/{proxy+} [OPTIONS GET HEAD POST PUT DELETE PATCH]

You can now browse to the above endpoints to invoke your functions.
You do not need to restart/reload SAM CLI while working on your functions,
changes will be reflected instantly/automatically. You only need to restart
SAM CLI if you update your AWS SAM template.

2018/03/11 15:12:46 Invoking my.service.StreamLambdaHandler::handleRequest (java8)
2018/03/11 15:12:46 Decompressing G:\Source\lambdas\my-service\target\my-service-1.0-SNAPSHOT.jar
2018/03/11 15:13:49 Mounting /c/Users/Daedin/AppData/Local/Temp/aws-sam-local-1520777566162251000 as /var/task:ro inside runtime container
START RequestId: 48dfc4ef-52c7-4674-8bf3-1853e7e8f4d4 Version: $LATEST
END RequestId: 48dfc4ef-52c7-4674-8bf3-1853e7e8f4d4
REPORT RequestId: 48dfc4ef-52c7-4674-8bf3-1853e7e8f4d4  Duration: 9083.56 ms    Billed Duration: 9100 ms        Memory Size: 512 MB     Max Memory Used: 15 MB
2018/03/11 15:14:01 Function returned an invalid response (must include one of: body, headers or statusCode in the response object): invalid character '-' after top-level value

I'm looking into this and trying to replicate. I'll update here as soon as I figure out what's going on.

hey @sebastientromp, The archetype works fine when deployed in Lambda. However, it breaks with SAM local. I suspect this has something to do with how SAM local reads the function output - it expects as the last message in the stdout. Spring Boot logs a lot of messages at boot and adds a shutdown message after the output is printed out. I thought this line was enough to switch off logging from Spring Boot, at least it was in the previous release. I'll do some more digging to figure out how to disable the spring boot logs.

Ok @sebastientromp, I figured out how to fix this for SAM local. Under /src/main, create a resources folder and put a new application.properties file in there. In the properties file we simply tell SpringBoot to turn off its logging (setting it to WARN may be enough).
logging.level.root=OFF

I'll keep this issue open to track the update of the archetype. I'll have this included in the next release.

Also ran into a variant of this issue. Running a clean checkout of the aws sample works fine, however as soon as you remove logback.xml (which sets the log level to ERROR), it fails with the same error. Creating the application.properties file with the logging.level.root=OFF (as mentioned by @SAPessi ) also fixes it in this scenario.

The following configuration in application.properties also works (only disables logging for the two packages that print at shutdown):
logging.level.org.springframework.jmx=ERROR
logging.level.org.springframework.boot.web.servlet=ERROR

Thanks @rafehi - I'll include this in the archetypes and samples for the next release

The fix for this is included in the latest merge. Closing the issue

Thanks! I'm just right now getting back to serverless, your help is much appreciated :)

Thanks @rafehi - I'll include this in the archetypes and samples for the next release

The doc already mentions archetype version 1.1, but it doesn't seem to exist yet. Am I correct?
I'll work with manual modifications in the meantime
Cheers!

Hey @sebastientromp, I can see the archetypes in maven central. What error are you getting?

Indeed, it works well. I must have made a typo when typing the commands.
Sorry for the noise, and thanks for the help!


Feature request - Explorer - Multiple file/folder select with cmd+click

Feature request - Explorer - Multiple file/folder select with cmd+click.
ala to Atom Tree View which allows multiple files/folder to be selected and dragged/copied/cut/pasted.

Duplicate #1023?

This issue has already been reported. Please use the search function next time before reporting an issue and/or this time if you are curious about the progress of the issue.


Sound from results

Generating sounds based on cellular automata involves a fascinating intersection of computer science, mathematics, and audio synthesis. Here's a step-by-step guide on how you could approach this project:
1. Understanding Cellular Automata
First, ensure you have a solid understanding of cellular automata (CA). Cellular automata are computational systems that evolve over discrete time steps according to a set of rules based on the states of neighboring cells. The most famous example is Conway's Game of Life.
2. Choose a Cellular Automaton
Decide which cellular automaton you'd like to use for sound generation. Different automata can produce vastly different behaviors, which in turn can influence the character of the sound. For a simple start, you could use the elementary cellular automaton or Conway's Game of Life.
3. Generate Cellular Automata Patterns
Implement or use an existing library to generate patterns with your chosen cellular automaton. You'll need to decide on the initial conditions and how many iterations you'd like to run. These patterns will be the basis for your sound generation.
4. Map CA Patterns to Sound Parameters
This step involves deciding how to translate the patterns from the cellular automaton into sound. There are many creative ways to do this, such as:

Mapping the state of each cell to a binary (on/off) sound event, like a note being played or not.
Using the number of active cells in each generation as a parameter to control aspects like pitch, volume, or timbre.
Interpreting the patterns as waveforms directly if you're looking for more abstract sounds.

5. Choose a Sound Synthesis Method
There are many sound synthesis techniques available, from simple waveform generation (sine waves, square waves, etc.) to more complex methods like FM synthesis or granular synthesis. The choice depends on the kind of sounds you're aiming for and the complexity you're willing to tackle.
6. Implement Sound Generation
Implement the sound generation logic in a programming language that supports audio processing. Python, with libraries such as Pyo or PySynth, could be a good choice due to its ease of use and the availability of libraries for both cellular automata and sound synthesis.
7. Experiment and Refine
Generating interesting sounds often requires a lot of experimentation. Play around with different cellular automata, mappings, and synthesis methods. Adjust parameters like the initial state, ruleset, and how exactly the CA patterns affect sound properties.
Tools and Libraries

Python: A versatile programming language with many libraries for both cellular automata and audio synthesis.

PyCellularAutomata: For generating cellular automata patterns.
Pyo, PySynth, or Tone.js (for web-based applications): Libraries for real-time audio synthesis and processing.

Example Workflow

Use a Python script to generate a pattern with the chosen cellular automaton.
Each row of the pattern represents a moment in time. Map this row to a series of notes or sound parameters.
Use an audio synthesis library to generate sound based on these parameters.
Iterate through the entire pattern, generating sound for each step.
Output the generated sound to a file or play it live.

Final Thoughts
The key to success in generating sounds with cellular automata is creativity in how you map the patterns to sound parameters and willingness to experiment. There's no single right way to do it, and the most interesting results often come from the most unexpected decisions.

Rule 30 is an elementary cellular automaton rule that exhibits chaotic behavior from simple initial conditions. When mapping each cell of a Rule 30 automaton to a sound parameter, consider the binary nature of the automaton's states: each cell can be either 0 (inactive) or 1 (active). Here are some creative ways to map these states to sound parameters:
1. Note On/Off
Each active cell (1) could trigger a note (e.g., a MIDI note or a specific frequency if you're synthesizing the sound), whereas inactive cells (0) could mean no sound is produced for that cell's position. This could result in a rhythmic pattern depending on the automaton's generation.
2. Pitch Mapping
Map the position of active cells within a generation to a musical scale or a set of frequencies. For example, the left-most cell in the automaton could correspond to the lowest note in your chosen scale, and the right-most cell to the highest note. Active cells then determine which notes are played at each step in the automaton's evolution.
3. Dynamic Parameters: Volume or Timbre
Use the density of active cells in a generation to control dynamic sound parameters. A higher density of active cells could correspond to a louder volume or a brighter timbre, while a lower density could result in softer or more mellow sounds. This approach could lead to interesting variations as the automaton evolves.
4. Spatial Mapping
If you're working with stereo or multi-channel audio, the position of active cells could be mapped to the stereo field or surround sound positions. For instance, cells towards the left of the automaton could pan sounds to the left speaker and vice versa, creating a spatial audio effect that evolves over time.
5. Generating Waveforms
Directly interpret each row in the cellular automaton as a binary waveform, where '1' represents a high amplitude and '0' a low amplitude. This can create complex and evolving waveforms that can be further processed or used as is.
Implementation Example in Python
Assuming you've generated a Rule 30 pattern and want to map it to sound parameters, here's a pseudocode outline:
import numpy as np
import your_synthesis_library

# Assume `ca_pattern` is a 2D numpy array where rows are generations
# and each cell's state is either 0 or 1
ca_pattern = generate_rule_30()

for generation in ca_pattern:
    for index, cell in enumerate(generation):
        if cell == 1:
            # Map cell position to a pitch
            pitch = map_position_to_pitch(index)
            # Generate and play note
            your_synthesis_library.play_note(pitch)
        # You could add else for handling inactive cells differently

    # Add a pause or delay here if needed, to separate generations

Tips for Experimentation

Scale and Harmony: Choose scales or sets of notes that lead to harmonious outcomes. Experiment with different mappings for varied results.
Rhythmic Variations: Adjust the timing between generations to create different rhythms.
Synthesis Techniques: Experiment with different synthesis techniques (subtractive, FM, granular) for a variety of timbres.
Effects Processing: Apply audio effects (reverb, delay, distortion) to the generated sounds for additional texture and depth.

Mapping cellular automaton states to sound parameters offers endless possibilities for creative sound design. Experimentation and iteration are key to discovering engaging and musically interesting mappings.


How to Init()?

Hi, I'm trying to POC this for storing events - however, I'm finding that I'm not able to get the init() call to complete.
I've got the initialisation code for the es in a file that is pulled into my index.js by a require statement. I can see the Init() line being called and it waits on the process.nextTick(function() { statement.
How should I be initialising the eventstore? My index.js spins up a REST api in express and listens to a port.

I need more infos to help you...
there is an example usage here: https://github.com/jamuhl/nodeCQRS


[Feature] Add two scripts for development assistance: login-pod/remote-debug-proxy

What is the purpose of the change
Improved development experience
Brief change log

Add two scripts for development assistance: login-pod/remote-debug-proxy
Change KinD cluster spec, make single node cluster as default

Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
Anything that affects deployment: (no)
The MGS(Microservice Governance Services), i.e., Spring Cloud Gateway, OpenFeign, Eureka.: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (docs)

LGTM


Shut down Heroku version of app

change the link to firebase on github repo

Link to firebase production version of app is now on GitHub repo.
Willy confirmed that the Heroku version of app has been shutdown and deactivated.


RFD - Extension Mechanism for Nebari

Status
Draft 🚧

Author(s)
@costrouc

Date Created
03-28-2023

Date Last updated
03-28-2023

Decision deadline
04-15-2023

Title
Extension Mechanism for Nebari
Summary
Over the past 3 years we have consistently run into the issue where extending and customizing Nebari has been a hard task. Several approaches have been added:

the addition of stages to the nebari deployment to make it easier to isolate pieces and was work that was done to make the extension mechanism easier to move towards
usage of terraform_overrides and helm_overrides keyword to allow for arbitrary overrides of helm values.
helm_extensions in stage 8 which allow for the addition of arbitrary helm charts
tf_extensions which integrate oauth2 and ingress to deploy a single docker image

Despite these features we still have needs from users and we are not addressing them all. Additionally we have issues when we want to add a new services it typically has to be directly added to the core of Nebari. We want to solve this by making extensions first class in Nebari.
User benefit
I see quite a few benifits from this proposal:

easier to extend Nebari making it easier to split development of nebari into smaller teams e.g. core Nebari team, feature-x team
easier customization of stages since the extension mechanism will solidify the interfaces between stages
easier adoption of new ways to deploy stages. Personally excited about this feature since it could make adoption of terraformpy easier.
ad-hoc client customizations will be significantly easier
ways to have proprietary additions to nebari that do not require deep customization

Design Proposal
Overall I propose we adopt pluggy. Pluggy has been adopted by many major projects including: datasette, conda, (TODO list more). Pluggy would allow us to expose a plugin interface and "install" extensions via setuptools entrypoints. Making extension installation as easy as pip install ...
Usage from a high level user standpoint
pip install nebari
pip install nebari-ext-clearml
pip install nebari-ext-helm
pip install nebari-ext-cost

Once a user installs the extensions we can view the installed extensions via:
$ nebari extensions list
Name                       Description
---------------------------------------------------------------------------
nebari-ext-clearml "ClearML integration into nebari"
nebari-ext-helm     "Helm extensions"
....

Plugin Interfaces
Within nebari we will expose several plugins:
Subcommands
A plugin interface for arbitrary additional typer commands. All commands will be passed in the nebari config along with all specified command line arguments from the user. Conda has a similar approach with typer for their system.
nebari cost

Stages
class Stage:
    name: str
    description: str
    priority: str    # defaults to value of name

    def validate(self, config: schema.NeabriConfig):
         """Perform additional validation of the nebari configuration specific to stage

         """

    def render(self, config: schema.NebariConfig) -> typing.Union[typing.Dict[str, bytes], pathlib.Path]:
          """Given a set configuration render a set of files

         Returns
         ------------
         typing.Union[typing.Dict[str, bytes], pathlib.Path]
              Returns either a directory to copy over files or a dictionary of keys mapping to file bytes
         """
         ...

      def deploy(self, directory: pathlib.Path, stages: typing.Dict[str, typing.Any]) -> typing.Any:
            "Deploy all resources within the stage

            "
            ...

      def destroy(self, directory: pathlib.Path):
            "Destroy all resources within the stage"
            ...

Nebari will use pluggy within its core and separate each stage into a pluggy Stage . Each stage will keep it's original name.
Alternatives or approaches considered (if any)
As far as plugin/extension systems go I am only aware of two major ones within the python ecosystem:

pluggy
traitlets :: I have used traitlets on several projects and do not feel it is a good fit here because:

traitlets is extremely invasive to the codebase it has opinions on the class structure/class creation
exposes a cli
opinionated way to perform customization

Best practices
This will encourage the practice of extending nebari via extensions instead of direct PRs to the core.
User impact
It is possible to make this transition seamless to the user without changing behavior.
Unresolved questions
I feel confident in the approach since I have seen other project use pluggy succefully for similar work.

Would it be possible to adopt this workflow with qhub-hpc and terragrunt deployment. Worth thinking how this extension mechanism could support this.

Implementing an extension mechanism to Nebari seems to be a very good solution for companies/peoples that wants to easily build and distribute features on top of Nebari 👍
We, at NaasAI, would like to expose:

Our needs around integrating with Nebari.
Requirements in terms of deployment management.
How we are doing it today
What we think would be a great way to interact with/use the Nebari CLI.

A bit of context
We are a company managing a JupyterHub infrastructure on top of Kubernetes and we have built features on top of JupyterLab to make it easier for users to schedule notebooks execution, share assets generated, build data products, etc ...
Integrating with Nebari seems like a good fit for us as we will be able to leverage what Nebari's team is doing while allowing us to focus on our core business.
Integrating with Nebari
We would like to distribute naas on top of Nebari.
For that, we must be able to customize Nebari's deployment, but also get information about Nebari's deployment to configure our resources accordingly. For example, we need to know:

What is the name of the cluster that was deployed?
In which region it was deployed?
What domain is used to serve nebari.
The name of the IAM role attached to EKS nodes.
...

Our requirements
In terms of infrastructure management, we think, at NaasAI, that Infrastructure as Code is a must, for multiple reasons that I won't explain here (unless you feel it would make this message clearer.), but a lot of companies would rather have an IaC solution than a manual process with a How to that an operator will need to follow (But it's not always the case, of course).
We want to be able to:

Manage multi-cloud deployments.
Manage multi-cluster deployment.
Separate infrastructure environments (dev/staging/preprod/prod/...) without limitations around the number of environments.
Integrate Nebari and Naas deployment into an existing IaC infrastructure.

How we are doing it today
We chose to use Terragrunt to manage our new infrastructure deployment as it seems to be a good fit for most use cases.
We have 3 repositories to manage our infrastructure deployment:

infrastructure-modules: This is the place where we create all our Terraform modules that will later be referenced/used to deploy infrastructure. (Terragrunt infrastructure-module example)
_envcommon: This one is used to have default values for our Terraform modules and also pinpoint specific terraform module versions. We have a configuration for each terraform module. (Terragrunt _envcommon example)
infrastructure-live: This one is used to reference configuration in _encommon and organize deployments dependencies and environments. We have the following logic where we split by environment/cloud_provider/region/project.  (Terragrunt infrastructure-live example)

To integrate with Nebari we tested multiple solutions but to be able to use our Terragrunt infrastructure we chose to create a Terraform Module allowing us to:

Pull terraform states generated by the nebari CLI when deploying the cluster.
Creating required terraform outputs that are later used by other Terraform modules using Terragrunt dependency mechanism.

This works for us today but has several drawbacks:

If Nebari changes something in its Terraform configuration it can break our Nebari state extraction module. Ex: A Terraform resource gets renamed then the JSON Path in the Terraform State will change and therefore break our module.
Distributing Naas becomes a bit more complicated because our users that want to deploy Naas will have to first deploy Nebari using the CLI, store all outputs safely somewhere, and then configure our nebari state extraction module with the right S3 bucket, S3 prefix etc. Even though this is deterministic, it's adding a step.
Our users only have one way to deploy Naas, by using our Terragrunt infrastructure setup, which is not a bad thing but can be quite complicated for testing purposes or smaller deployments (even though we could figure a simpler Terragrunt deployment method where all needed resources could be embedded in a single repo, not recommended for long term use though).

What we think would be a great way to interact with/use the Nebari CLI.
A big strength of the Nebari CLI, that we think is worth keeping, no matter what is done next, is the fact that it's very easy to deploy a Nebari infrastructure. You just need to have to run npm install nebari and follow the 2-step deployment to, configure your deployment and then actually deploy infrastructure.
This will fit a lot of user needs.
On the other hand, there is a need to help users/companies that wants to deploy Nebari and customize it using IaC to do it.
Regarding the fact that, as of today, Nebari CLI is a wrapper around Terraform, to order and split deployments into multiple stages, we could definitely have a parameter to maybe not actually deploy Nebari infrastructure using the CLI but rather use it to template/render Terragrunt configuration for example (Terragrunt being one of many templating outputs that we could have, Terraform, Pulumi, AWS CDK and so on).
To give an example, let's say I want to deploy Nebari in our Naas Terragrunt infrastructure. Let's assume that we need to deploy in a dev environment, on aws in us-west-1.
I would like to be able to do:
# Create the needed folder structure to match our way to split deployment.
mkdir -p infrastructure-live/dev/aws/us-west-1/nebari

# Go to Nebari folder for dev/aws/us-west-1 deployment.
cd infrastructure-live/dev/aws/us-west-1/nebari

# Install Nebari CLI
conda install nebari -c conda-forge

# Configure my deployment
nebari init --guided-init

# Generate Terragrunt configuration (Usually it would be the time to run "nebari deploy -c nebari-config.yaml")
nebari generate -c nebari-config.yaml --template-to=Terragrunt

Then if I were to execute the tree command in the infrastructure-live directory I would get the following output:
.
└── dev
    └── aws
        └── us-west-1
            └── nebari
                ├── infrastructure
                │   └── terragrunt.hcl
                ├── kubernetes-ingress
                │   └── terragrunt.hcl
                ├── kubernetes-initialize
                │   └── terragrunt.hcl
                ├── kubernetes-keycloak
                │   └── terragrunt.hcl
                ├── kubernetes-keycloak-configuration
                │   └── terragrunt.hcl
                ├── kubernetes-services
                │   └── terragrunt.hcl
                ├── nebari-config.yaml
                └── nebari-tf-extensions
                    └── terragrunt.hcl

12 directories, 8 files

Now it would be up to me to deploy the infrastructure by running for example:
cd dev/aws/us-west-1/nebari
aws-vault exec <myprofile> -- terragrunt run-all apply

So now what happens with pluggy extensions?
This would allow us to build our own nebari-naas pluggy extension and use it like so:
cd dev/aws/us-west-1/nebari
pip install nebari-naas

nebari extensions nebari-naas generate -c nebari-config.yaml --template-to=Terragrunt

This would then have generate a new directory with it's Terragrunt configuration that could have Nebari previous stages as dependencies.
Example:
.
└── dev
    └── aws
        └── us-west-1
            └── nebari
                ├── infrastructure
                │   └── terragrunt.hcl
                ├── kubernetes-ingress
                │   └── terragrunt.hcl
                ├── kubernetes-initialize
                │   └── terragrunt.hcl
                ├── kubernetes-keycloak
                │   └── terragrunt.hcl
                ├── kubernetes-keycloak-configuration
                │   └── terragrunt.hcl
                ├── kubernetes-services
                │   └── terragrunt.hcl
                ├── nebari-config.yaml
                ├── nebari-naas 👈
                │   └── terragrunt.hcl 👈
                └── nebari-tf-extensions
                    └── terragrunt.hcl

13 directories, 9 files

Then if we want to have multiple naas extensions as well we could distribute:
pip install nebari-naas
pip install nebari-naas-extension-aaa
pip install nebari-naas-extension-bbb
pip install nebari-naas-extension-ccc

Conclusion
Here I mainly talked about the fact to generate Terragrunt configuration, but I think that this should just be an additional way of being able to deploy Nebari and extensions. Nebari and extensions should also be able to be deployed solely using the Nebari CLI.
I tried to give as much information as possible, but please, if something is not clear enough or you feel that it needs more explanation, tell me and I will try to make it clearer.
I am looking forward to having the possibility to deploy Nebari in a very modular way while still complying with most users/companies needs in terms of deployment strategy. I think that this extension mechanism can be a very strategic move for the adoption of Nebari and the growth of its ecosystem.


Documentation about API for keys and pool methods.

@ankurdotb Looks like there will be several ADRs that are cheqd related but about VDR Tools. What is the right place for them?

@ankurdotb Looks like there will be several ADRs that are cheqd related but about VDR Tools. What is the right place for them?

I don't see this document as an ADR, it's more a developer documentation for how to use the software. Good question on whether it should be here though: if it's about how to use VDR Tools, then I'd expect it to be im VDR Docs but we can link to those docs from ours. Is there doc site similar to docs.cheqd.io for VDR Tools?


[HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() // to branch-0.x

Change Logs
In HoodieTableMetaClient remove getBasePathV2() and return StoragePath from getBasePath().
(This PR is to branch-0.x as @yihua asked me here https://github.com/apache/hudi/pull/11385#pullrequestreview-2095238250)
Impact
none
Risk level (write none, low medium or high below)
none
Documentation Update
none

The config description must be updated if new configs are added or the default value of the configs are changed
Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the instruction to make
changes to the website.

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

@hudi-bot run azure

@hudi-bot run azure

Azure CI is green.


Reference model alignment with the current policy

Hello,
I've been exploring the implementation of Proximal Policy Optimization (PPO) in the ppo_trainer.py file, and I have a query regarding the handling of the reference model (old policy) in relation to the current policy model.
In standard PPO implementations, it's a common practice to periodically update the reference model so that it aligns with the current state of the policy model. This approach ensures that the policy model's deviations are measured against a relatively recent version of itself, aiding in stabilizing the training process and preventing the policy from diverging too drastically or quickly. However, upon reviewing the PPOTrainer class in the provided code, I couldn't locate the segment where this periodic alignment of the old policy (reference model) with the current policy (model itself) is performed.
I'm wondering if this might be an intentional design choice in this particular implementation, or if there's a possibility that I might have overlooked some aspect of the process. Could it be that the updating of the reference model is handled implicitly in another part of the codebase, or is it the case that the reference model remains static throughout the training process in this implementation?
Thanks,

I believe when fine-tuning the pretrained LLMs the reference model is already very strong and the policy model should never deviate to far from it. This is very different from pure RL and I think that's the reason/intuition why the reference model is not updated in this setting usually.

@lvwerra thanks so much for your insights; it does make sense in the case of LLMs. This actually raises a compelling question in my mind: what about scenarios involving less potent language models, specifically those of a smaller scale, like those with merely hundreds of millions of pre-trained parameters?
If we can make sure this smaller LM gets expertise on the downstream task (possibly through metrics in the validation steps), is it a good idea to make this update/alignment with the current policy happen?
Thanks in advance,

I think the main reason is that one doesn't really expect the model (even a small one) to get better at language modeling than it was after pretraining. The PPO step in general is just to align better with some preference. If you think it's holding the model back you can always weaken the KL term which will decrease the coupling between the reference and active model.
Of course you are free to test it out yourself, maybe this is all wrong :)


Parameterize Paket/FAKE binaries used in build.sh

A previous commit (b612328) moved the location of FAKE.exe around but
missed updating every reference to it in build.sh. (This is why the Mono build is broken in CI).
This should make sure that doesn't happen again.

Thx


Opacity conflict issue

Hi there. Thank you so much for making this package. It saved my day. It's exactly what I needed. I was already going crazy with Stack approach.
However I would like to report a bug, or maybe it's more of a improvement.
When you set barrierColor property to some solid, non-transparent color (eg. black) then barrierOpacity property makes sense to apply. But if I set barrierColor to some color that already have opacity in its alfa channel (eg. black54), then that exact opacity value should be used for drawing barrier and animation for it. It that case, barrierOpacity property should be ignored. Otherwise, we are applying opacity over opacity.

Hey @giga10, glad this helped you! Thanks for opening this Issue, will make sure to tackle it as soon as I can.


alpine-base + openjdk

I'd like to see an openjdk-based image using alpine-base.
My team works with the Play Framework, which is perfectly capable of running in the standard "docker way" - except that the openjdk/alpine image doesn't have the s6 overlay, and thus I believe will suffer from the resolv.conf issue, and thus could benefit from the go-dnsmasq functionality you have here.
If there's interest, I will fork and submit a PR.

Hey @rbellamy, that sounds great. I'd like to see something Java related in here. I don't use it much so don't have the expertise but if you're keen to add it, that'd be great.
It is worth noting and considering however that the latest version of Alpine, 3.4, has fixed all of the resolv.conf issues so go-dnsmasq is technically not required anymore.

Well... that's interesting. I didn't realize that. That's really my forcing function here, so I think I'll put this on the back burner for now. Thanks for the heads-up!

@rbellamy, no worries, I'll close this issue for now but you're welcome to reopen or start a new one should you become interested in this again.

I'm a bit confused by something I found in https://github.com/nginxinc/docker-nginx/issues/78. Looks to me like the expected fix in v3.4 of alpine hasn't resolved the problem... am I reading that right?


Update instructions for Kubernetes deployment of ODIM + URP, GRF and Dell plugins deployment

Adding instructions for Kubernetes deployment of ODIMRA.
Adding instructions for URP, GRF and Dell plugins deployment + their addition to ODIM.
Instructions are documented in the main readme, and the relevant plugin instructions are linked from the main readme to the readme present in the respective plugin dirs.

Changing Title to include Kubernetes deployment of ODIMRA

Closing this PR because it is no longer valid. Had files other than the ones relevant to documentation. Raised a new PR for this and got the content approved and merged to upstream dev branch. Hence closing this PR.


CouchbaseContainer does not actually wait until the query service is available

It seems that even though the CouchbaseContainer checks the health of the /admin/ping endpoint for the query service, the query service is not actually available to handle queries yet.
If I withReuse(true) this container, and run the test a second time, it succeeds because it now seems the query service has had enough time to initialize and become available to handle queries.
2020-07-14 13:26:20.936  INFO 80842 --- [           main] 🐳 [couchbase:enterprise-6.5.1]          : Creating container for image: couchbase:enterprise-6.5.1
2020-07-14 13:26:21.051  INFO 80842 --- [           main] 🐳 [couchbase:enterprise-6.5.1]          : Starting container with ID: 1ece82baa13e1965812f1371320d97b75e92fb2aab727246dd5e3ccba984a3d4
2020-07-14 13:26:21.629  INFO 80842 --- [           main] 🐳 [couchbase:enterprise-6.5.1]          : Container couchbase:enterprise-6.5.1 is starting: 1ece82baa13e1965812f1371320d97b75e92fb2aab727246dd5e3ccba984a3d4
2020-07-14 13:26:21.640  INFO 80842 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /youthful_dirac: Waiting for 60 seconds for URL: http://localhost:33429/pools
2020-07-14 13:26:21.661  INFO 80842 --- [tream-142261320] docker.couchbase                         : STDOUT: Starting Couchbase Server -- Web UI available at http://<ip>:8091
2020-07-14 13:26:21.661  INFO 80842 --- [tream-142261320] docker.couchbase                         : STDOUT: and logs available in /opt/couchbase/var/lib/couchbase/logs
2020-07-14 13:26:31.307  INFO 80842 --- [     ducttape-0] o.t.c.wait.strategy.HttpWaitStrategy     : /youthful_dirac: Waiting for 30 seconds for URL: http://localhost:33429/pools/default
2020-07-14 13:26:31.959  INFO 80842 --- [     ducttape-0] o.t.c.wait.strategy.HttpWaitStrategy     : /youthful_dirac: Waiting for 30 seconds for URL: http://localhost:33427/admin/ping
2020-07-14 13:26:32.970  INFO 80842 --- [           main] 🐳 [couchbase:enterprise-6.5.1]          : Container couchbase:enterprise-6.5.1 started in PT14.344S
2020-07-14 13:26:32.977  INFO 80842 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /youthful_dirac: Waiting for 60 seconds for URL: http://localhost:33429/pools/default/buckets/YS_Apps
2020-07-14 13:26:34.124  INFO 80842 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /youthful_dirac: Waiting for 60 seconds for URL: http://localhost:33429/pools/default/buckets/YS_Secure
2020-07-14 13:26:35.105  INFO 80842 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /youthful_dirac: Waiting for 60 seconds for URL: http://localhost:33429/pools/default/buckets/YS_Ephemeral
2020-07-14 13:26:36.145  INFO 80842 --- [           main] 🐳 [couchbase:enterprise-6.5.1]          : Couchbase container is ready! UI available at http://localhost:33429

2020-07-14 13:26:38.041  INFO 80842 --- [           main] com.couchbase.client.core.CouchbaseCore  : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslTruststoreFile='null', sslKeystorePassword=false, sslTruststorePassword=false, sslKeystore=null, sslTruststore=null, sslHostnameVerificationEnabled=false, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=33429, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=33422, bootstrapCarrierSslPort=11207, ioPoolSize=12, computationPoolSize=12, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=12, queryServiceEndpoints=12, searchServiceEndpoints=12, configPollInterval=2500, configPollFloorInterval=50, networkResolution=NetworkResolution{name='auto'}, ioPool=NioEventLoopGroup, kvIoPool=null, viewIoPool=null, searchIoPool=null, queryIoPool=null, analyticsIoPool=null, coreScheduler=CoreScheduler, memcachedHashingStrategy=DefaultMemcachedHashingStrategy, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.7.11 (git: 2.7.11, core: 1.7.11), retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, continuousKeepAliveEnabled=true, keepAliveErrorThreshold=4, keepAliveTimeout=2500, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, callbacksOnIoPool=false, disconnectTimeout=25000, requestBufferWaitStrategy=com.couchbase.client.core.env.DefaultCoreEnvironment$4@4f654cee, certAuthEnabled=false, coreSendHook=null, forceSaslPlain=false, compressionMinRatio=0.83, compressionMinSize=32, compressionEnabled=true, operationTracingEnabled=true, operationTracingServerDurationEnabled=true, tracer=ThresholdLogTracer, orphanResponseReportingEnabled=true, orphanResponseReporter=DefaultOrphanResponseReporter, keyValueServiceConfig=KeyValueServiceConfig{minEndpoints=1, maxEndpoints=1, pipelined=true, idleTime=0}, queryServiceConfig=QueryServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, searchServiceConfig=SearchServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, viewServiceConfig=ViewServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, analyticsServiceConfig=AnalyticsServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, queryTimeout=7500, viewTimeout=7500, searchTimeout=75000, analyticsTimeout=75000, kvTimeout=2500, connectTimeout=5000, dnsSrvEnabled=false, propagateParentSpan=true}
2020-07-14 13:26:38.686  INFO 80842 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Connected to Node localhost
2020-07-14 13:26:38.737  INFO 80842 --- [-computations-8] c.c.c.core.config.ConfigurationProvider  : Selected network configuration: external
2020-07-14 13:26:38.746  INFO 80842 --- [-computations-8] c.c.c.core.config.ConfigurationProvider  : Opened bucket YS_Apps
2020-07-14 13:26:38.752  INFO 80842 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Disconnected from Node localhost
2020-07-14 13:26:38.771  INFO 80842 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Connected to Node <IP_ADDRESS> (localhost)

org.springframework.dao.DataAccessResourceFailureException: The Query service is not enabled or no node in the cluster supports it.; nested exception is com.couchbase.client.core.ServiceNotAvailableException: The Query service is not enabled or no node in the cluster supports it.

	at org.springframework.data.couchbase.core.CouchbaseExceptionTranslator.translateExceptionIfPossible(CouchbaseExceptionTranslator.java:87)
	at org.springframework.data.couchbase.core.CouchbaseTemplate.execute(CouchbaseTemplate.java:551)
	at org.springframework.data.couchbase.core.CouchbaseTemplate.queryN1QL(CouchbaseTemplate.java:505)

I have a feeling this might be a CB Java SDK issue now, I added extra checks (running some n1ql queries) to the CouchbaseContainer right after startup, the same checks I added to our integration tests pre test start.
They pass from CouchbaseContainer but fail for about 1-4 seconds from the integrationt tests.
I'll dig deeper into this, perhaps turn on client side debug in the CB Java SDK and see try and see what is going on

Yeah this appears to be an issue with the CB Java SDK.
But out of the box this prevents the use of the CouchbaseContainer for N1QL queries...
2020-07-15 10:14:07.879 DEBUG 60242 --- [           main] c.couchbase.client.core.RequestHandler   : Got instructed to add Node localhost
2020-07-15 10:14:07.879 DEBUG 60242 --- [           main] c.couchbase.client.core.RequestHandler   : Connecting Node localhost
2020-07-15 10:14:07.881 DEBUG 60242 --- [           main] c.couchbase.client.core.RequestHandler   : Connect finished, registering for use.
2020-07-15 10:14:07.884 DEBUG 60242 --- [-computations-5] c.couchbase.client.core.RequestHandler   : Got instructed to add Service BINARY, to Node localhost
2020-07-15 10:14:08.308  INFO 60242 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Connected to Node localhost
2020-07-15 10:14:08.355  INFO 60242 --- [-computations-8] c.c.c.core.config.ConfigurationProvider  : Selected network configuration: external
2020-07-15 10:14:08.355 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got notified of a new configuration arriving.
2020-07-15 10:14:08.355 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Starting reconfiguration.
2020-07-15 10:14:08.356 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Starting reconfiguration for bucket YS_Apps
2020-07-15 10:14:08.356 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Node <IP_ADDRESS>
2020-07-15 10:14:08.356 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Connecting Node <IP_ADDRESS>
2020-07-15 10:14:08.357 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Connect finished, registering for use.
2020-07-15 10:14:08.357 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Service BINARY, to Node <IP_ADDRESS>
2020-07-15 10:14:08.358 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Service VIEW, to Node <IP_ADDRESS>
2020-07-15 10:14:08.360 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Service CONFIG, to Node <IP_ADDRESS>
2020-07-15 10:14:08.363  INFO 60242 --- [-computations-8] c.c.c.core.config.ConfigurationProvider  : Opened bucket YS_Apps
2020-07-15 10:14:08.364 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Starting reconfiguration.
2020-07-15 10:14:08.364 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Starting reconfiguration for bucket YS_Apps
2020-07-15 10:14:08.364 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Node <IP_ADDRESS> already registered, skipping.
2020-07-15 10:14:08.364 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Service BINARY, to Node <IP_ADDRESS>
2020-07-15 10:14:08.365 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Service VIEW, to Node <IP_ADDRESS>
2020-07-15 10:14:08.365 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to add Service CONFIG, to Node <IP_ADDRESS>
2020-07-15 10:14:08.365 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Removing and disconnecting node localhost.
2020-07-15 10:14:08.365 DEBUG 60242 --- [-computations-8] c.couchbase.client.core.RequestHandler   : Got instructed to remove Node localhost
2020-07-15 10:14:08.370  INFO 60242 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Disconnected from Node localhost
2020-07-15 10:14:08.389  INFO 60242 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Connected to Node <IP_ADDRESS> (localhost)
2020-07-15 10:14:10.091  INFO 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : Started AbstractScopedEntityDAOIT in 28.218 seconds (JVM running for 29.13)
2020-07-15 10:14:10.106 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.209 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.315 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.416 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.520 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.621 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.727 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.832 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:10.934 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.038 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.140 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.243 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.347 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.451 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.556 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.658 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.761 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.866 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:11.967 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.071 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.177 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.280 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.383 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.484 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.585 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.686 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.793 ERROR 60242 --- [           main] c.y.r.u.d.d.i.AbstractScopedEntityDAOIT  : The Query service is not enabled or no node in the cluster supports it.
2020-07-15 10:14:12.831 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Got notified of a new configuration arriving.
2020-07-15 10:14:12.831 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Starting reconfiguration.
2020-07-15 10:14:12.831 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Starting reconfiguration for bucket YS_Apps
2020-07-15 10:14:12.831 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Node <IP_ADDRESS> already registered, skipping.
2020-07-15 10:14:12.831 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Got instructed to add Service QUERY, to Node <IP_ADDRESS>
2020-07-15 10:14:12.832 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Got instructed to add Service BINARY, to Node <IP_ADDRESS>
2020-07-15 10:14:12.832 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Got instructed to add Service VIEW, to Node <IP_ADDRESS>
2020-07-15 10:14:12.833 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Got instructed to add Service CONFIG, to Node <IP_ADDRESS>
2020-07-15 10:14:12.833 DEBUG 60242 --- [-computations-9] c.couchbase.client.core.RequestHandler   : Got instructed to add Service SEARCH, to Node <IP_ADDRESS>

Created a CB support ticket #35272 to track this

cc @daschl

@aaronjwhiteside would it be possible for you to enable TRACE level and supply those logs? The reason is the error you are seeing means that the config the client got from the server does not include query, and right now I have a hard time figuring out the timeline of events. If you could send the trace over, it should include the raw configs the client gets from the server and maybe this helps figuring out what's going on.

@daschl here you go client-trace.txt
Thanks for looking into this!

@daschl this second trace includes more details, like when the server actually decides to push the config containing the query service.
client-trace2.txt

@aaronjwhiteside thanks for sharing the log, I'll check it out and get back to you

@aaronjwhiteside this is really interesting. 2 bucket configs arrive at the client, the first one does not have the query service enabled but the second one does. I'm not sure why this is the case, because the container code enables all services before it starts creating a bucket.
Can you post your code and how you are using it so I can give it a try to reproduce? Also, which SDK version are you using?

Java SDK 2.7.11, with Spring Data Couchbase 3.1.2 which modifications. I've also tried with CB 6.0.3, 6.5.0 and 6.5.1 there is no difference between them.
Our setup is fairly custom/esoteric both in terms of TestContainers and the way we configure Spring Data Couchbase... so it's not easy to provide the code. I'll work on trying to provide a minimum test case that reproduces the issue..

@daschl Ok turns out to be really simple to reproduce..
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>test-containers-couchbase-issue-2993</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.couchbase.client</groupId>
            <artifactId>java-client</artifactId>
            <version>2.7.11</version>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>couchbase</artifactId>
            <version>1.14.3</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.30</version>
        </dependency>
    </dependencies>
</project>

import com.couchbase.client.java.Bucket;
import com.couchbase.client.java.Cluster;
import com.couchbase.client.java.CouchbaseCluster;
import com.couchbase.client.java.env.CouchbaseEnvironment;
import com.couchbase.client.java.env.DefaultCouchbaseEnvironment;
import com.couchbase.client.java.query.N1qlQuery;
import org.testcontainers.couchbase.BucketDefinition;
import org.testcontainers.couchbase.CouchbaseContainer;

public class Example {
    public static void main(final String...args) {
        try (final CouchbaseContainer container = new CouchbaseContainer()
                .withBucket(new BucketDefinition("bucket1")
                                    .withPrimaryIndex(false))) {
            container.start();

            final CouchbaseEnvironment environment = DefaultCouchbaseEnvironment
                    .builder()
                    .bootstrapCarrierDirectPort(container.getBootstrapCarrierDirectPort())
                    .bootstrapHttpDirectPort(container.getBootstrapHttpDirectPort())
                    .build();

            final Cluster cluster = CouchbaseCluster.create(environment, container.getHost())
                    .authenticate(container.getUsername(), container.getPassword());

            final Bucket bucket = cluster.openBucket("bucket1");
            bucket.query(N1qlQuery.simple("SELECT 1 FROM system:completed_requests WHERE true"));
        }
    }
}

@aaronjwhiteside pr #3003 should fix the issue. I think as a workaround in the meantime you can either sleep or re-enable the primary index building which will also "take some time" which increases the chance the config is in the state the client needs.

Keep alive!


Can we have some keyboard shortcuts?

Hi. Thanks again for the tool, I use it a lot.
Since I spend a lot of time using the Create window, can we please have some kind of keyboard shortcuts?
Or maybe the kind where I can use the Alt key combined with an underlined letter to select a box or use a function?
Hoping we can have this in the next update if possible. Thank you.

That's probably doable. If you want to map out exactly which shortcuts would do what, I can look into adding them.

Give this version a try and see what you think https://github.com/Ephemerality/xray-builder.gui/releases/tag/2.1.189

Thank you for your quick update!
I've been using it and so far so good. I'll let you know if there's any issue in the future.

Hi. I completely forgot that Ctrl + Del can be used to delete a whole word in a field, so now it's conflicted with the Remove button.
Could you please help me change the Ctr + Del to Alt + Del?
And the Ctrl + Shift + Del to Alt + Shift + Del?
Thank you.

Sure, try https://github.com/Ephemerality/xray-builder.gui/releases/tag/2.1.190