Model trained in AWS fails in Win10 machine(Both AWS/WIN10 has same Keras/TF versions)

I am training a model in AWS and while using the saved model from AWS to WIN10 simulator, i am getting this error. The same clone.py runs without issues in WIN10 model.
Please can you help.
C:\Users\abhijith.bhat\OneDrive - Accenture\UdacityDLND\CarND\CarND-Behavioral-Cloning-P3>python drive.py model.h5
Using TensorFlow backend.
Traceback (most recent call last):
  File "drive.py", line 122, in <module>
    model = load_model(args.model)
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\models.py", line 239, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\models.py", line 313, in model_from_config
    return layer_module.deserialize(config, custom_objects=custom_objects)
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\layers\__init__.py", line 54, in deserialize
    printable_module_name='layer')
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\utils\generic_utils.py", line 139, in deserialize_keras_object
    list(custom_objects.items())))
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\models.py", line 1214, in from_config
    model.add(layer)
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\models.py", line 442, in add
    layer(x)
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\engine\topology.py", line 602, in __call__
    output = self.call(inputs, **kwargs)
  File "C:\Users\abhijith.bhat\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\layers\core.py", line 650, in call
    return self.function(inputs, **arguments)
  File "clone.py", line 42, in <lambda>
    model.add(Lambda(lambda x: (x / 255.0)-0.5 , input_shape=(160,320,3)))
IndexError: tuple index out of range

https://github.com/bhatabhijithn/Car-Behaviour-Cloning

If you get to know the issue, please let me know

I have same issue, do you have any idea?

I had the same issue. In my case, I found that the error occurs because of python version inconsistency. My local python version was 3.5.5 and the remote was 3.6.3. After installing same python version with the remote the model can be loaded without any error.


[bug] Type issue on networks with createAppKit from the example

Link to minimal reproducible example
https://github.com/Tbaut/react-sprinter/blob/main/src/AppKitProvider.tsx
Summary
When using the example from the appkit dashboard, there's a type issue on networks:

Type '({ blockExplorers: { readonly default: { readonly name: "Etherscan"; readonly url: "https://sepolia.etherscan.io"; readonly apiUrl: "https://api-sepolia.etherscan.io/api"; }; }; contracts: { ...; }; ... 9 more ...; serializers?: ChainSerializers<...> | undefined; } | { ...; } | { ...; })[]' is not assignable to type '[AppKitNetwork, ...AppKitNetwork[]]'.
Source provides no match for required element at position 0 in target.ts(2322)
TypesUtil.d.ts(43, 5): The expected type comes from property 'networks' which is declared here on type 'CreateAppKit'

// 3. Set the networks
const networks = [sepolia, b3Sepolia, baseSepolia]

// 4. Create Wagmi Adapter
const wagmiAdapter = new WagmiAdapter({
  networks,
  projectId,
  ssr: false
})

// 5. Create modal
createAppKit({
  adapters: [wagmiAdapter],
  networks, // <------------------------------- The type error appears here
  projectId,
  metadata,
  features: {
    analytics: false // Optional - defaults to your Cloud configuration
  }
})

List of related npm package versions
   "@reown/appkit": "^1.6.0",
   "@reown/appkit-adapter-wagmi": "^1.6.0",

Node.js Version
v22
Package Manager
<EMAIL_ADDRESS>

please check this example -> https://github.com/reown-com/appkit-web-examples/blob/main/react/react-wagmi/src/config/index.tsx

thanks type assertion as [AppKitNetwork, ...AppKitNetwork[]] solves the issue.


enable the addition of listeners/notifications on value changes to update other elements of a GUI

Possibility of extending this by adding an interface, something along the lines of Updateable, with a method onUpdate, called on the object whenever the GUI is updated. Should just be able to check if the object implements this interface, and calling that method inside whatever listener will be added. I attempted this myself but I don't know enough about javafx to know which listener to add, and I don't know enough about how your code works to add the listener myself.

Added 2 new Interfaces in the new release for this.


Display Error in Chrome

Hi Mr McBride
I have been having some issues displaying the GEOJSON's in Google chrome. When I open the page the loading bar displays over the map and the GEOJSON's fail to display, in addition the lists are not searchable as it appears not to be reading any of the GEOJSON content. It works perfectly in Firefox but I would like to be able to use it on other web browsers, eventually in an offline networked environment pointing to an map server.
I would greatly appreciate any advice you may have.
Thank you!!
Kesh

Do you have a live link you can share for debugging? Are you getting any JavaScript errors in Chrome?

Hi Mr Mcbride
I have been working to correct this problem for a while and the day after i message you I find information as detailed in the link below. I now have it working, I apologise for my preemptive query.
Thank you  for your time and quick response.
https://code.google.com/p/chromium/issues/detail?id=40787
Cheers
Kesh!
Date: Sat, 27 Jun 2015 06:08:08 -0700
From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>CC<EMAIL_ADDRESS>Subject: Re: [bootleaf] Display Error in Chrome (#26)
Do you have a live link you can share for debugging? Are you getting any JavaScript errors in Chrome?
—
Reply to this email directly or view it on GitHub.

Good to hear!


useAxiosRepo with typescript - Argument of type 'typeof MyModel' is not assignable to parameter of type 'Model'

Hello,
i am using the axios plugin in a typescript project to retrive data from an api.
Running the code in dev mode works. But it won't compile:
Argument of type 'typeof MyModel' is not assignable to parameter of type 'Model'.
  Type 'typeof MyModel' is missing the following properties from type 'Model': _meta, $self, $entity, $config, and 38 more.

await useAxiosRepo(MyModel).api().get('mymodels', {

Maybe someone has a suggestion for me
Regards
nowrap

The useAxiosRepo definition is:
declare function useAxiosRepo<M extends Model>(model: M): AxiosRepository<Model>;

And the useRepo definition is:
declare function useRepo<R extends Repository>(repository: R | Constructor<R>, pinia?: Pinia): R;
declare function useRepo<M extends Model>(model: Constructor<M>, pinia?: Pinia): Repository<M>;

Any solution or workaround to this , it seems to work while still showing the error.

It should be fixed in the edge channel and later in the 1.9.0 release. Feel free to test it out.
Feedback always welcome.

Fixed with #816


Win 10 installation problem

Hi, author. Thanks for supplying so wonderful code. I want to use it on my win 10 operation system. When I installed it, I met an error, like this:
-- Configuring incomplete, errors occurred!
See also "D:/paper/human-pose-estimation/pose_extractor/build/tmp/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "setup.py", line 72, in
cmdclass={'build_ext': CMakeBuild})
File "D:\Anaconda\lib\site-packages\setuptools_init_.py", line 145, in setup
return distutils.core.setup(**attrs)
File "D:\Anaconda\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "D:\Anaconda\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "D:\Anaconda\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "D:\Anaconda\lib\site-packages\setuptools\command\build_ext.py", line 84, in run
_build_ext.run(self)
File "D:\Anaconda\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "D:\Anaconda\lib\distutils\command\build_ext.py", line 340, in run
self.build_extensions()
File "setup.py", line 63, in build_extensions
subprocess.check_call(['cmake', ext.cmake_lists_dir] + cmake_args, cwd=tmp_dir)
File "D:\Anaconda\lib\subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', 'D:\paper\human-pose-estimation\pose_extractor', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=D:\paper\human-pose-estimation\pose_extractor\build', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY_RELEASE=D:\paper\human-pose-estimation\pose_extractor\build\tmp', '-DPYTHON_EXECUTABLE=D:\Anaconda\python.exe', '-DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE', '-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE=D:\paper\human-pose-estimation\pose_extractor\build', '-DCMAKE_GENERATOR_PLATFORM=x64']' returned non-zero exit status 1.
Can you help me to solve it? Thanks!

how to execute step 3 in win10?

Hi! This says, that error happened during cmake invocation, the real error should be above the line -- Configuring incomplete, errors occurred!. Will you post the full log?

Hi, author. The first problem I have solved. I don't know how to do step 3 on win 10? Where to place build folder? Which and where the folder? Thanks !

Can you please post your solution? There is similar issue in #4. For the step 3 you should have build folder inside pose_esimator folder (this done automatically during running setup.py build_ext). After that, just update environmental variable PYTHONPATH, for windows type in command line:
set PYTHONPATH=pose_extractor\build;%PYTHONPATH%

Hi, author
I had build folder in pose_estimator. My solution is: edit a new variable in system environmental variable. But it didn't work. I tried your solution, too. Type above line in cmd. But it also didn't work.

Did you still have the same error above? Can't guess without real error.

Sorry for not supplying error before, author. I still have this error. The error like this:
Traceback (most recent call last):
File "demo.py", line 10, in
from modules.parse_poses import parse_poses
File "D:\pycharm\teacher_analyse\human-pose-estimation\modules\parse_poses.py", line 4, in
from pose_extractor.src import extract_poses
ImportError: DLL load failed: 找不到指定的模块。
I followed your instructions in readme.md. Step 1 and 2 with no error. But when do step 3, I met this error. Can you kindly tell me what I had done is wrong through my error? Thanks!

My problem is in .py file. It seems cannot directly call cpp file and hpp file.

You should have from pose_extractor import extract_poses, instead of pose_extractor.src. Yes, python cannot call c++ code from source. So, first run python setup.py build_ext, it should make a build folder inside pose_estimator and make there pose_estimator.so. Then set PYTHONPATH and you are ready to run the demo.

I'm also on Windows. I did this
set PYTHONPATH=pose_extractor\build;%PYTHONPATH%
With the demo script, there's an error on line 4 of parse_poses.py. I get
from pose_extractor import extract_poses
ImportError: DLL load failed. The specified module could not be found.
Then I renamed pose_extractor.pyd to pose_extractor.dll
The error is the same location but different:
ImportError: cannot import name 'extract_poses'
If I just enter python and do
import pose_extractor
there are no errors. I have a feeling extract_poses is sort of "missing."
>>> pose_extractor.__dict__
{'__path__': _NamespacePath(['C:\\Users\\david.braun\\Documents\\GitHub\\lightweight-human-pose-estimation-3d-demo.pytorch\\pose_extractor']), '__loader__': <_frozen_importlib_external._NamespaceLoader object at 0x000001CA0D9FD5C0>, '__spec__': ModuleSpec(name='pose_extractor', loader=None, origin='namespace', submodule_search_locations=_NamespacePath(['C:\\Users\\david.braun\\Documents\\GitHub\\lightweight-human-pose-estimation-3d-demo.pytorch\\pose_extractor'])), '__name__': 'pose_extractor', '__doc__': None, '__package__': 'pose_extractor'}
Should extract_poses have appeared in that dictionary?

Hi, author
I ran setup.py and I didn't make .so file in my pose_estimator folder. The problem is same.

pose_estimator.so should be in pose_estimator\build folder. And you should have from pose_extractor import extract_poses, instead of pose_extractor.src in modules/parse_poses.py.

Hi, author
I know all steps of instructions. But there is not a .so file in build folder. Below is my compile log in cmd. Is that right?
Compile log:
D:\TSBrowserDownloads\human-pose-estimation>python setup.py build_ext
running build_ext
-- Building for: Visual Studio 16 2019
-- The C compiler identification is MSVC 19.22.27905.0
-- The CXX compiler identification is MSVC 19.22.27905.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx64/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: D:/Anaconda/python.exe (found suitable version "3.7.3", minimum required is "3.5")
-- Found PythonLibs: D:/Anaconda/libs/python37.lib (found suitable version "3.7.3", minimum required is "3.5")
-- Found OpenCV: D:/opencv/opencv/build (found suitable version "4.1.2", minimum required is "4")
-- Configuring done
-- Generating done
-- Build files have been written to: D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/build/tmp
用于 .NET Framework 的 Microsoft (R) 生成引擎版本 16.2.37902+b5aaefc9f
版权所有(C) Microsoft Corporation。保留所有权利。
Checking Build System
CMake does not need to re-run because D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/build/tmp/CMakeFiles
/generate.stamp is up-to-date.
Building Custom Rule D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/CMakeLists.txt
CMake does not need to re-run because D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/build/tmp/CMakeFiles
/generate.stamp is up-to-date.
wrapper.cpp
extract_poses.cpp
human_pose.cpp
peak.cpp
正在生成代码...
Auto build dll exports
正在创建库 D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/build/tmp/pose_extractor.lib 和对象 D:/TSBrowserDownl
oads/human-pose-estimation/pose_extractor/build/tmp/pose_extractor.exp
pose_extractor.vcxproj -> D:\TSBrowserDownloads\human-pose-estimation\pose_extractor\build\pose_extractor.pyd
Building Custom Rule D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/CMakeLists.txt
CMake does not need to re-run because D:/TSBrowserDownloads/human-pose-estimation/pose_extractor/build/tmp/CMakeFiles
/generate.stamp is up-to-date.

I use python which intergrated in anaconda. Actually, there is no parameter named pythonpath  in variable environment. So, in my situation, how to install?

I use msvc in vs 2019.

I mean .pyd library suffix. Set environmental variable PYTHONPATH, type in command line:
set PYTHONPATH=pose_extractor\build;%PYTHONPATH%

Then try in python:
import pose_extractor
dir(pose_extractor)

it should list extract_poses, if not, then something went wrong.

Hi, author
I have tried everything I can do to solve this problem. I also followed your instructions above. But, it still had ImportError: DLL load failed: 找不到指定的模块.

Don't know how to help. Updated the code, now it can run without c++ module. Check it out!

Hope, it helped.

Now, it can work! Thanks!

I have the exact same error.
How can I get it to work without c++ module?

Just run it. It will use pure python implementation automatically.

Does anyone know this error?  ,Thanks
My OS：Windows 10
I followed the instructions of Github,and I get file at <pose_extractor\build\pose_extractor.pyd>, but when I executed it, I encountered：
#### Cannot load fast pose extraction, switched to legacy slow implementation. ####
Then I print out the wrong paragraph, I get：
from pose_extractor import extract_poses
ImportError: DLL load failed: 找不到指定的模組。


Reduce extra queries on Show Name observations menu using a PORO

coverage: 94.569% (+0.005%) from 94.564%
when pulling 3ba1776945d9e95d7f62b33aedc08422c73731c3 on nimmo-name-observations-menu
into d7f9472f3ae7113b6ae0d8d7e384b69d91d1136c on nimmo-panels-show-name.


Events page

DOD: have a dropdown menu that displays various groups and clubs. when the user selects one of them, it loads the ads for that event

Does this issue differ from
Issue 29: dod: user can click on an event on the event page and view more information on a separate page. have links for google maps and to enable notification.


Retrieve Private messages

This is no issue, just a question.
Which of the functions should I use to retrieve chat messages?
I tired with
$friends = $snapchat->getFriends();
 getConversationInfo($friends)
But I get an enormous stdClass mixed with arrays.

solved:
$friends = $snapchat->getFriends(); $conversationInfo= $snapchat->getConversationInfo($friends); //print_r($conver); //echo "/////////////////////////////////////////////////////////////"."\n"; //print_r($conver["0"]["sasstv"]->conversation_messages); $conversation =""; foreach ($conversationInfo as $A) {  	foreach ($A as $B) {  		if(!isset($B->conversation_messages->messages))   		{    			break;   		}    		$C=$B->conversation_messages->messages;   		foreach($C as $D) {  			if(!isset($D->chat_message->header->from))   			{    				break;   			}   			$conversation .= "From: ";  			$conversation .= ($D->chat_message->header->from)."\n";  			$conversation .= "To: ";  				foreach ($D->chat_message->header->to as $E) {  					$conversation .=$E."\n";  				}  			$conversation .="Message: ";  			$conversation .= ($D->chat_message->body->text)."\n"."\n";      	} 	} } return $conversation;

solved in https://github.com/mgp25/SC-API/pull/196

@mgp25 hey, I need some help to make this function better, were is the right place for me to get some help from you?

@LanchaBasualdo email


Do we want Board representation?

This is a spin-off discussion from https://github.com/nodejs/bootstrap/pull/12.
I would like to ask everyone in this group what kind of Board representation do we want or need. Currently the project has 3 Board seats: 1 CommComm Director, 1 TSC Director, 1 individual. The Board is currently composed of 10 members.
In a multi-project foundation, our current setup might not work.
Some questions from @rubys:

Do we want only Node.js to have board representation?
Do we want each and every project, no matter how big or how small, to have board representation?
Do we want to have a single person to represent all projects?

If none of the above, what exactly do we want?
To this regard, I would kindly ask @MylesBorins @hackygolucky @jasnell @rvagg (as the directors that have been elected by the TSC/CommComm) to share their experience as directors.

The merger poll that I sent out to the TSC and CommComm is not quite completed yet, but we have 24 responses out of 34 possible (there are actually 26 responses but 2 non top level committee folks voted and I'll be removing their responses from the results when completed) ... that said, here are the current results on this specific question:

I think that makes it pretty clear. 84.6% of respondents believe Node.js must have board representation, with the largest majority of those strongly agreeing.

I think in terms of board voting rights, the projects should be represented and it was probably a mistake of the JSF to not do this as it led to the perception of less transparency and less of a voice for the projects. The preceding Dojo and jQuery foundations both had substantial project representation on the board fwiw.
I would argue for multiple board seats for the projects. 1 for Node.js given its size, and 1 or 2 to represent the other projects via some sort of vote from the leads of all of the projects (I think 1 person to represent 30-some projects including Node.js is a bit much to ask of that one person, though it could be achieved), and maybe others depending on how things like CommComm and TSC end up.
Or there might be an alternative where projects are sorted into a few groups (strawman: Node.js gets grouped with things like NodeRed and IoT/embedded projects, the various client-side JS projects get grouped together, and maybe the testing+linting projects in a third group, and from that you get 3 groups to each vote on a board rep).
In terms of non-voting rights (e.g. option to attend but do not vote), in terms of transparency I think you open that up to one person from each project.
The follow-up question that may get asked (or at least which I don't know the answer to): Do the board meetings happen online or in person. If the latter, that may impact the ability for the non-member to afford attending meetings unless the foundation is expected to cover travel costs.

Currently the project has 3 Board seats: 1 CommComm Director, 1 TSC Director, 1 individual. The Board is currently composed of 10 members.

I count 11.  Not that the actual number matters all that much.
I sense apportioning fractional directors up front across projects is likely to be a problem, so how about splitting this into two parts?
First part: determining the number of directors that come from the projects.  Perhaps this could be a fixed ratio of the corporate members to project members?  Say, start with the number of corporate members and divide by three and round up (or perhaps divide by two and round down).  This number to be reevaluated at a fixed time (perhaps annually).
Second part: determining who to elect.  Perhaps a foundation wide election where all collaborators of any project is eligible to cast a single vote?  If a process like [STV}(https://wiki.apache.org/general/BoardVoting) is used, large projects will get their proportional share but generally not much more.  I encourage viewing and reading the links in the "How Are Votes Counted?" section of that page.

First part: determining the number of directors that come from the projects. Perhaps this could be a fixed ratio of the corporate members to project members? Say, start with the number of corporate members and divide by three and round up (or perhaps divide by two and round down). This number to be reevaluated at a fixed time (perhaps annually).

I like this approach. This is simple.

Second part: determining who to elect. Perhaps a foundation wide election where all collaborators of any project is eligible to cast a single vote? If a process like STV is used, large projects will get their proportional share but generally not much more. I encourage viewing and reading the links in the "How Are Votes Counted?" section of that page.

This is where things gets tricky. I like an approach where large projects will get their share, but not much more. I'll need to study STV (which I didn't know).
Also, who can be nominated?

Currently there are 12 board members, 3 of which are community representation.

One of the challenges from the JSF side is the range of scales of each project. Projects do not have any direct representation on the board. Projects that have meet a set of criteria can graduate out of 'mentorship' at which point they get to nominate a representative to the Technical Advisory Committee. The Chair of the TAC sits on the Board - which does mean, in the current structure, one person represents all 28 projects on the board - albeit indirectly via the TAC.  Of the 28 projects, only 2 have graduated and thus have representation on the TAC - leaving 26 projects without any representation.
The criteria for graduation are focussed on the size of the project's governing body and the diversity (in terms of organisations) of that body. It makes passing reference to other criteria in the project's original application, but doesn't spell out what they are. Some projects were grandfathered into the JSF and some we added as part of the foundation launch. As one of the latter, I have no idea what other criteria apply to Node-RED.
If we want board representation to be reflective of a project's size, we need very clear criteria what we mean by size.

Is it the number of committers?
Is it the size of its governance body?
Is it some popularity metric?

Is a project that is maintained by a single developer but gets 10m downloads a week a bigger or smaller project than one that has 3+ different companies involved in its governance (as per graduation criteria), but only has 1000 downloads a week.
There is also the fundamental question of whether node.js should be considered 'just one of the projects' or as an exceptional entity within the foundation. I would suggest that, by pretty much any metric chosen, node.js would be considered a 'big' project and earn whatever representation that garners. That would keep everything on level playing field.

I'm concerned that we're not looking at the problem in the right light in order to evaluate these proposed solutions. For the record, I'm not against project representation on the board, but I struggle to see how any of the options submitted thus far will prevent the issues that we've all raised on other threads. I think there's another side to the 'board seat' problem. Hear me out:
Projects want a seat at the board table because it guarantees that someone from the project has access to information from the board and a voice on the board to raise issues or concerns with the $Foundation.
This is a very real need, but as @mcollina pointed out it doesn't scale - as the number of projects grow,  it becomes increasingly difficult to schedule meetings, reach quorum, and ensure everyone is familiar enough with an issue to take a vote on it. Further, a lot of the board's proceedings are going to be agnostic to the concerns of a given project (should we sponsor a conf? should this new project come in? should we renew our vendor relationship with LF? etc - while those items would be of interest to folks, they're also probably not something that projects will feel strongly enough about to weigh in on,  except in extreme circumstances).  We also get into the issue of what happens when a project representative on the board also happens to work at a member company who already has a seat at the board. Keeping track of the rules and relationships here could get tricky. I don't know of another software foundation or consortia that provides a seat at the board for every project, so there's not a lot of prior art I'm aware of for that organizational model.
The communication flow between the foundation boards and their projects has been less than ideal, and is the primary cause of concern during these merger conversations.
That said, both foundations ostensibly had board representation through different means. The JSF TAC chair is a member of the JSF board, and Node's TSC, CommComm & Individual directors are members of the Node Board. The communication issues were present despite some degree of access; so why is that? To briefly summarize my perception of that from the JSF side:

Some projects expected and wanted more consistent engagement, which did not match the structure or resource limitations of the JSF.
The JSF relied on a 'nested hierarchy' for info to roll up to the board from projects via the TAC.
The TAC chairperson, who is not paid by the foundation, has a busy full-time job outside of the day-to-day of the project communities, and focused their time on mentoring and assisting projects as needed.
Almost everyone in the org is a volunteer with different projects - our day-to-day is focused on our communities, not the whole org. And also our jobs, which may be unrelated to our projects.
The JSF ED, the foundation's only employee, was directed to focus on recruiting new members and mentoring a few new projects, leaving little time/bandwidth for existing projects.
Regular updates to the projects wasn't made anyone's job. It was assumed that JSF news would reach people via social media channels or the occasional email thread.

The result was a lot of mis-set expectations and unmet needs, and it needs to be said that none of this was intentional or malicious. All of the points listed above can be easily solved - they just need to be addressed directly & transparently. I encourage the community directors from Node to think about why, despite having board seats, the community doesn't feel heard by the board. I imagine some of this is the same and some of it is different.
Project board seats are an implementation detail; the problem it's meant to solve is communication and expectation-setting.
So we could make some kind of arrangement where projects over a certain size get board seats, and maybe others are grouped in some capacity (how do you measure that, as @knolleary pointed out?) - but I fear we end up with the "trickle-down info" problem we had in the JSF, where not everyone was truly being represented.
Or we could also have an omsbuds-person for every project, or a group of projects, who is invited to attend board meetings and whose sole job is to report and advocate for their project.
Or we could create a 'Reporter' officer position on the Board, whose job it is to do this legwork and make sure information is shared and gathered.
Or we could create job descriptions for the "board seats" and have a people who serve on the board for their project commit to specific tasks for a certain term.
Or we could do all of the above.
We're all very busy people, and good communication is really going to be everyone's job. Let's make sure the new system we create makes this as easy as possible and doesn't re-create the same issues.

(Is there a "Foundations for Dummies" book?  Or even a TL;DR version thereof?  In other words...excuse my ignorance.)
I'm trying to understand why I'd want representation on a board.

Where do the board's responsibilities begin and end?
Could the responsibilities be minimized to such an extent that a seat would be of little use to a project?  Or is this not possible, since the board has influence upon the ED?
Shouldn't the projects be working more with the foundation and its employees/volunteers itself, not its board?

@knolleary

The Chair of the TAC sits on the Board - which does mean, in the current structure, one person represents all 28 projects on the board - albeit indirectly via the TAC. Of the 28 projects, only 2 have graduated and thus have representation on the TAC - leaving 26 projects without any representation.

This is more or less the situation that I would like to avoid. IMHO every project should have a direct line up and their member should feel represented and part of something bigger.

Well written @jorydotcom!
I think we have one of our first problem statements that we would like to solve:

Projects want a seat at the board table because it guarantees that someone from the project has access to information from the board and a voice on the board to raise issues or concerns with the $Foundation.

Project board seats are an implementation detail; the problem it's meant to solve is communication and expectation-setting.

Considering the fact that a significant amount of the work the Board do happens at closed doors, I fear the only way to guarantee that flow of information is to have a proper seat at the table. The TSC and CommComm do not have visibility on the 100% that gets discussed by the Board (some details are extremely confidential): we trust our representatives to act on behalf of the project. I fear that without some votes allocated to the projects, our voices will not be heard, and I know that all of our directors have been instrumental in growing the Node.js project because of their role. This has been far from perfect, but it works.
Even from a pure formal point of view, having a vote could send a meaningful signal.

@boneskull  It's perfectly OK for a project to not want to be involved, and to not consume any Foundation resources. Express and Liibuv falls into this category.
If a projects needs anything from the Foundation to keep growing, it's fair to assume that they would like to check that their needs are met year after year if the funds are available. Especially if they cannot leave / or gather funds elsewhere.
The more funds a Foundation gathers, the more they can do things for the projects, the more the projects would like to check how that money is being spent. Bad marketing can be more harmful than no marketing at all. If the foundation cannot do anything for the projects - then why raising funds in the first place?

@mcollina my sense is that you are misunderstanding @boneskull question, and perhaps bringing in some of your own perspective - not that there is anything wrong with that, but I would like to tease apart the real question that we need to address.
Let's start with @jasnell's graph.  I read it as "without a common understanding of what the role of the board is, do you wish to be a part of the board (either directly or by proxy)".
My sense (by talking to people who are not on the board) is that the perception is that the board is where the action is (or at the very least, where the money flows from).  My experience (from being on a different board, and from the few board members that I have talked to) is that this either is not the case, or should not be the case.
This comes down to: is the board the place where decisions are made, or is it a place where decisions are ratified?  Is the board more like a loan officer or an ATM?  Is the money "their money" but if you make the case, they will begrudgingly let you have access to some?  Or is the money "your money" and the board's role is to ensure that you presented the right credentials and don't go beyond your limits?
In the ASF, we just kicked off our annual five year plan review, where I will be gathering input from each of the budget holders (generally non-directors, and those that happen to be directors will be contributing to this discussion with the Director "hat" taken off).  I will be leading this process, and am not currently a Director, and in fact, intentionally stepped down as Director when I toke on the role as President.
Early next year, we will follow this up with annual budgets.
A different model.  One that appears healthier.  One where people who want something from the foundation deal not with the board but with somebody who is pulling together a budget.  And once that budget is approved, work with accounting (again not the board).  The board's role in this is to approve the budget (mostly ATM style, as in did the people with the right credentials participate in the process?), and to review monthly reports from the Treasurer as to whether spending is going according to plan.
Circling back (and potentially putting words into @boneskull's mouth, but I don't think so give that what I believe I am saying is consistent with the words he posted): does wanting to be a beneficiary of being a part of this foundation require direct involvement with the board?

Let's start with @jasnell's graph. I read it as "without a common understanding of what the role of the board is, do you wish to be a part of the board (either directly or by proxy)".

My experience has been that Node.js top level committee members tend to have a higher than average understanding of what is happening at the board level than most. The desire to have representation on the board is not about being where the "action is" as we all have a fairly solid understanding that the decisions that directly affect the project happen at the collaborator level. It's about making sure the project has input into larger issues around the brand, legal, and financial issues when they arise.

understanding of what is happening at the board level

I focus on the word "is" in that sentence.  There seems to be a lot of angst over what "is" happening.  Perhaps there may be a better alternative?

It's about making sure the project has input into larger issues around the brand, legal, and financial issues when they arise.

I read into this a presumption that these issues are perceived to be owned by the board.  Am I reading this correctly?  Would perhaps a better organization be one where these are spun off into separate functions with autonomy, where the board's role of one of oversight instead?
At the ASF, VP Brand happens to be a board member (though I will assert that his role as VP Brand is separate from his role as Director).  We currently have two people sharing the role of VP Legal, one is a Director and one in not.  Neither the ASF Treasurer nor VP Finance are Directors, though the Assistant Treasurer is.
I continue to assert that there is a different organizational model that may be a better fit for this proposed merged foundation.  All of the ASF board minutes are published online, and I can walk people though this if there is interest.  I've also created a lot of automation around the process that I can help bring here.

@rubys are 100% of ASF Board meetings public?

@mcollina short answer, no, but damn close.
Longer answer:
Every ASF Officer or ASF Member can attend any board meeting at the ASF.  These don't have direct analogized here, but every project has a chair which is an officer (and here is another case of where projects recommend a person to be chair and the board routinely ratifies that person, it is exceedingly rare for the board to name or replace a chair, and in those rare cases it is an exceptional circumstance).  ASF members also don't have an exact analogy here, it generally is a level above collaborator, so members of TAC or TSC and ConCom are a close match).  To give you an idea, we have 198 projects (each with an officer) and 730 members (clearly there is overlap between those two sets).  We also have various other officers, though pretty much all of them are members.
Second, the full agenda (which includes all officer reports) is available to all officers and members.  Per the above, that's over 800 people.  A small portion (as in 5% or less) is redacted.  This generally means things like the names of perspective sponsors, names of groups with which we have trademark issues, and names of perspective collaborators that have yet to be named.
Finally, there is something rare and exceptional, called executive session. That's where the board has a private meeting.  These tend to happen less than once a year, mostly to do things like electing executive officers (which sometimes is done privately, other times openly), and rarely (as in a once or twice a decade) to handle sensitive matters.  Most of these sessions are on the order of 15 minutes, an hour on the outside.

if the project were allowed more direct oversight over the use of the brand, that would also make it significantly less important to sit on the Board.

That's exactly the type of input we need.  Thanks!
My sense is that project representation doesn't scale, and any attempt to impose a rank ordering of projects (example: platinum vs gold vs silver) is counter productive.  That being said, I sense little to no pushback to having a percentage of board members being elected at large, and I can bring knowledge and tooling to solving that problem.
What does scale is delegation. The ASF has a Brand Management committee that reports not to the board but to the president.  The roster for that committee is stored in https://svn.apache.org/repos/private/committers/board/committee-info.txt, which isn't public, but is visible to the 6549 people who have commit rights to an ASF repository.  The Brand Management committee currently has 9 members, only two of which are current Directors.
I'll close by noting that Brand Management as a committee is not called out in the ASF bylaws.  What is described is the ability to create committees (and similar language is present in both the JSF and Node.js bylaws).  So this is an ability that the ASF makes productive use of, and is essentially "unexpressed DNA" in the Node and JS Foundations.

Some insight from a current board member.
We currently have both a private and public session when we do meetings currently. Both sessions are an hour. The public session is broadcast on youtube but can only be attended by board members, LF staff, and invited guests.
We attempt to put as much in the public session as possible. The private session tends to be used for confidential discussion. At times this confidential discussion can be shared with the project, and board reps such as myself, @hackygolucky, or @williamkapke will officially ask the board for permission to share the results of the private discussion. Examples of this would be a high level view of a program while not discussing the budget.
The board does have subcommittees to discuss certain matters, and these subcommittees meetings are generally private. The main subcommittees I can think of at the moment are Legal, Financial, Marketing, and the subcommittee formed to discuss the merger.
My experience has been that if / when these things can be made public they will be, as you see with the merger subcommittee joining the bootstrap team. That being said some topics cannot be talked about with the same candor in a public setting. This is a tradeoff. As such many early ideas that need to be discussed with extreme candor can be introduced in a private setting and moved to a public setting at their earliest convenience.
Board members have expressed in the past a desire to create public sub-committees that are empowered by the board to make decisions, have a budget, and have community representation.
I personally believe that if we were to empower both a Cross Project Council (technical) and a C3 (community) with decision making authority and budget we would significantly reduce the need for direct project board representation. I'll even take it a step further and say that we likely should have foundation staff who's full time job it is to ensure that we execute on our decisions (a project / program manager).
Even if we were able to get the board representation we ask for, the time commitment is fairly large if you want to be an effective board member. The real long term solution to this would be to empower the projects to have budget, to be able to make decisions, and to have the foundation supply the adequate resources to make sure we execute. I don't think it is fair or strategic to rely on volunteers to execute on these important matters.

Reactions to @MylesBorins comment above:

I agree that the public/private is a tradeoff.  That being said, my sense is that the current "default closed" assumption of Node.js (as evidenced by the need to ask permission to share) vs a "default open" assumption (everything is sharable unless otherwise noted, the model for the ASF board minutes) does not serve the Node.js foundation well at this time.

The right model might not be for projects to have budgets but for departments (trademarks, marketing, conferences, etc) to have budgets, and to have projects participate in those committees.

volunteers vs paid staff isn't as straightforward as you suggest.  I know some amazing volunteers who have been around for decades and continue to consistently contribute.  I know some paid staff that were let go, hired away, or resigned after months.  The closest I have come to a conclusion is that the split isn't between important and not-so-important, as between urgent and not-so-urgent.  If something needs a timely response, there should be paid staff responsible for that function.  There are many important functions that aren't urgent that can be handled by motivated volunteers.

@rubys response to response

The default is to put things in the public session. When things are in the private session that is the only time we need to ask to share.

I was thinking it more like a larger cross project council had a budget and projects could request it... but I think we are getting into implementation details here... glad we are on the same page about it not being 'board business'

I agree with everything you stated. I was more thinking about things in terms of responsibility. That the execution on important matters should be the responsibility of a foundation employee whose performance review and job is dependent on executing.

Sorry, I can't resist injecting a little levity into this discussion.  ;-)
So node.js releases should be made by foundation employees?  Or node.js release aren't important?  Or...?
I'm old enough to remember when the conventional wisdom was that you couldn't trust open source as it was developed and maintained by volunteers.
P.S.  I still think that the public/private tradeoff choices that are being made need reexamining.

The urgent versus non-urgent resonates with me.  When an action needs immediate response it's a harder fit for volunteers.  It is hard for a volunteer to have the expectation to always be available and hard on those who are waiting for something urgent.
On the other hand we are very successful in trusting volunteers with very important areas/activities.

Thanks everyone.
@rubys is correct in interpreting my awkwardly-worded question.  @mcollina It's not that I, as a project owner, don't want to leverage foundation resources; it's rather a question of "what kind of board would I want representation on?"
From the comments above, if the board's responsibilities and power are diminished, and projects are in control of their trademarks and destiny, and budgets are no longer "board business", then it sounds like a project owner wouldn't necessarily want to bother with board representation.
@jasnell's chart shows that project members want a seat at the board's table.
It doesn't answer the question of why.  Presumably, because that's where you eat if you're working with the current Node.js foundation.  :wink:

After talking with folks in Vancouver and also reading some replies, it occurs to me that we may want to disambiguate what kind of Foundation we're talking about here. I'm picking up on some competing ideas of what a Foundation is so hopefully this is helpful context for folks (apologies to anyone for whom this info is not new):
The LF, Node, and JS Foundations are 501(c)(6) organizations that exist to promote the common interests of its members in open source software (or professional sports, or dentistry, or whatever the shared interest is). In practice this means marketing, branding, and PR work for key technologies, and for members' investment in said technologies (via case studies, certification, thought leadership, etc).
When most people think of a non-profit foundation, they think of a 501(c)(3), which are tax-exempt charitable organizations that operate in the public interest. ASF & other, older open source software foundations are (c)(3)s, but the IRS has since changed its mind on that.*
In the latter case, Board members are usually expected to raise a certain amount of funds or to contribute a certain amount of their own money to the foundation. In cases where the foundation takes public donations, there is often a board requirement that there are neutral advisory parties (people who don't stand to benefit in anyway) who are responsible for ensuring that the money is handled appropriately. Board membership here requires active engagement.
In the former case, the board membership pattern both our organizations currently have is common practice - people or organizations pay money to become members of a group, associate their brand with that group, and promote or benefit from group membership in some (non- or indirectly commercial) way. Lots of times, board seats in these kinds of orgs are expected to be fairly procedural & not particularly active.
It seems like our community is looking for a blend of both approaches:

We want a board that is hands-off project work & direction, provides input when needed, and will hold the new ED & their staff accountable for delivering value to members + projects. I think this aligns with the expectations of the current boards as well.
We also want a board that is fairly distributing resources of the organization to the projects as needed (travel funds, promo items, etc) and/or available to assist in personnel issues (a tricky CoC violation, for example). I think it's hard for the board to play this role without being more engaged in the projects, and is not what the current boards want to be doing.

If I understand @boneskull & others correctly - they want to work with organization staff on the items in 2) and have a clear path to provide feedback for 1) without the meeting obligations of a board seat. If decisions around project funding and branding/promotion are moved out of the Board's work into separate top-level committees, and projects have clear representation on those groups, does that strike the right balance of @mcollina @jasnell @boneskull et al?
personal opinion here: it would be great to write a purpose statement for the new org that had some public benefit language, and then see if we can make the case for becoming a (c)(3)....

I have way less concern about board representations if:
a. Node.js could potentially leave the foundation, if its leadership decides so.
b. The use of the Node.js brand is not limited to the foundation personnel.
c. Funds are managed with project involvement, and allocating XYZ to do ABC does not require a Board meeting (within limits).
d. Important decisions for the project are done at project level, not at the Board.
As @jasnell said, control over the Node.js brand has been used in the past to control its contributors. I think this is a situation that everybody that was (or was not) involved at that time would like to repeat.
Based on the agreements done with Joyent when the Foundation was created, I do not think a) or maybe even b) is feasible for Node.js. This alone makes it a strong requirement for Board representation. It also makes Node.js a special snowflake within the multi-project foundation.

If I understand @boneskull & others correctly - they want to work with organization staff on the items in 2) and have a clear path to provide feedback for 1) without the meeting obligations of a board seat. If decisions around project funding and branding/promotion are moved out of the Board's work into separate top-level committees, and projects have clear representation on those groups, does that strike the right balance of @mcollina @jasnell @boneskull et al?

@jorydotcom Overall I think it strikes the right balance, and we should work towards that goal independently of Board representation. I believe Board representation is a must unless the above point gets resolved.

@mcollina fantastic context, thank you. That arrangement is different than what the projects have at JS Foundation. JSF holds some, but not all, of the project marks and doesn't weigh in on project branding issues except to help legally protect the mark (like the time we had someone doing some inappropriate things with the Grunt logo) and help order stickers from time to time. Our projects would expect that strategy to continue, and to be true for all projects I think.
@MylesBorins @mhdawson @hackygolucky: Could we get someone (or multiple someones) from the board to weigh in on these questions (slightly rephrased from Matteo)? Or maybe we can discuss them at the next meeting?

a. Could Node.js or any foundation project leave the foundation, if its leadership decides so?
b. Will the use of the Node.js or any foundation project brand be limited to the $Foundation or its personnel?
c. Will projects be able to co-manage its funds, such that allocating XYZ to do ABC for the project does not require a Board meeting (within limits)?
d. Does the Board expect to be able make important decisions for specific projects (beyond accepting it into the foundation and ensuring its compliance with policies)?

There's probably some gray areas with all of these questions, but it would be ideal to confirm and capture the intent of the boards so we can reason about this more.

I would put one small spin on question (a) above:
(A) Could Node.js or any foundation project leave the foundation, it is leadership decides so, and continue to use their associated brand. Specifically, if Node.js decided, at any point, to leave the Foundation, could it continue to irrevocably use the Node.js name and mark?

Meta comment: this is our (collective) foundation, not the board's.  We should endeavor to act like we own it.  By that, I mean that we should be focusing on proposing solutions, not asking questions.

Could Node.js or any foundation project leave the foundation, if its leadership decides so?

First obvious observation: the code is open source, so forks can happen.  And people aren't chattel, so they can contribute where they wish.  So the question really comes down to brand.  And I see that as I composed this reply, @jasnell came to the same conclusion.
There are limits to which a 501(c)(6) can give away property which have commercial value (and brands clearly qualify here), but I don't think that will be the sticking point.
My sense is that even if the answer to the first question is "yes", that won't satisfy.  I think we need a concrete proposal as to how this could occur, and what it would involve.

Will the use of the Node.js or any foundation project brand be limited to the $Foundation or its personnel?

Trademark law is complex, and this question has many layers.  There are parts where we need active participation of a lawyer who is well versed in trademark law.  There are parts that are best done by the projects themselves.  I would suggest Apache PMC Branding Responsibilities as a potential starting point.

Will projects be able to co-manage its funds, such that allocating XYZ to do ABC for the project does not require a Board meeting (within limits)?

I think it is time for these Foundations to start producing an annual budget where owners have allocated funds that they can use within the scope of their responsibility during the course of the year.

d. Does the Board expect to be able make important decisions for specific projects (beyond accepting it into the foundation and ensuring its compliance with policies)?

My sense is that this question is too open ended.  In general, a board has few limits on what they can do.  For example, a board can decide to "retire" a project due to inactivity.  But a board that routinely acts that way will negatively impact the longevity of the foundation.  The normal model should be that projects and other committees propose and the board ratifies.

From conversations I’ve been privy to I believe that any project can leave the foundation, but the IP / trademark stuff would need to be transferred to another 501(c)6 or 501(c)3. Someone please chime in if I’m incorrect.
Node.js is in a unique situation as the foundation does not of the trademark, so they would be unable to transfer it.

On Oct 24, 2018, at 10:50 AM, Sam Ruby<EMAIL_ADDRESS>wrote:
Meta comment: this is our (collective) foundation, not the board's. We should endeavor to act like we own it. By that, I mean that we should be focusing on proposing solutions, not asking questions.
Could Node.js or any foundation project leave the foundation, if its leadership decides so?
First obvious observation: the code is open source, so forks can happen. And people aren't chattel, so they can contribute where they wish. So the question really comes down to brand. And I see that as I composed this reply, @jasnell https://github.com/jasnell came to the same conclusion.
There are limits to which a 501(c)(6) can give away property which have commercial value (and brands clearly qualify here), but I don't think that will be the sticking point.
My sense is that even if the answer to the first question is "yes", that won't satisfy. I think we need a concrete proposal as to how this could occur, and what it would involve.
Will the use of the Node.js or any foundation project brand be limited to the $Foundation or its personnel?
Trademark law is complex, and this question has many layers. There are parts where we need active participation of a lawyer who is well versed in trademark law. There are parts that are best done by the projects themselves. I would suggest Apache PMC Branding Responsibilities http://www.apache.org/foundation/marks/responsibility as a potential starting point.
Will projects be able to co-manage its funds, such that allocating XYZ to do ABC for the project does not require a Board meeting (within limits)?
I think it is time for these Foundations to start producing an annual budget where owners have allocated funds that they can use within the scope of their responsibility during the course of the year.
d. Does the Board expect to be able make important decisions for specific projects (beyond accepting it into the foundation and ensuring its compliance with policies)?
My sense is that this question is too open ended. In general, a board has few limits on what they can do. For example, a board can decide to "retire" a project due to inactivity. But a board that routinely acts that way will negatively impact the longevity of the foundation. The normal model should be that projects and other committees propose and the board ratifies.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nodejs/bootstrap/issues/17#issuecomment-432689874, or mute the thread https://github.com/notifications/unsubscribe-auth/AAecV7IGsj-zVsUIYQ8Tr3fqo6ZcdVkwks5uoH49gaJpZM4XzICq.

I've opened a separate thread to discuss the trademark question: https://github.com/nodejs/bootstrap/issues/25

@mcollina w/r/t trademark and Joyent, are you alluding to needing to rename the project “io.js”? Or is there something else I’m missing? don’t want to derail this but just wanted to make sure I understood the context.

@boneskull I think the whole question will be answered on issue #25.

I will add that the story of Node.js and Joyent is one of if not the most important reason why open source foundations matter. There was some discussion earlier around what the purpose of a foundation is. It's important to have a neutral home for projects before a project becomes big and issues like this become contentious where one company tries to exert control and direction over a project.
For what it's worth (not much really), it's why we started the Dojo Foundation pretty much at the inception of Dojo in 2004 (the other reason being we didn't want to get personally served with legal notices like we were with the predecessor netWindows projects :) ), and why we encouraged so many projects over the years to join the effort, even projects that were considered by some to be direct competitors to existing foundation projects (I think it's silly to think of properly licensed and managed OSS projects as competitors because we share the same bigger goals).
For many years and even today, people doubt the value of software foundations, but Node.js and MySQL are just two of many examples where an OSS project has had tumultuous times when a company overexerted its control over a project. In my opinion this could have been avoided if the project trademark and copyright were placed in a foundation early on. It's too late to avoid that painful portion of Node.js' history, but hopefully the foundation can save future JS projects from similar struggles.

From the last meeting it seemed like we had consensus across various stakeholders that we want to maintain community representation, but that it was a bit premature to figure out how that representation will be implemented until we understand the governance structure of the larger foundation. Some specific notes worth considering

Proportional representation

currently 25% of our board is community reps
this is something that can be codified in the bylaws

likely anchored to platinum memerbship

Prior art

the CNCF currently has a similar number of community reps on their board
a board seat is reserved for the largest project in the foundation

I'm going to remove the bootstrap-agenda label for now, until we have a bit of a better understanding of the foundation governance process. If anyone disagrees with this please re-add the label or ask me to


When manually editing Framework References via the Metadata xml editor it's possible to crash the applciation

Type BUG
NPE version: (e.g. 5.x) (see Help -> about)
OS version:  Windows 10 1942
Installed from: Built from Source

What is the current result?
If you manually edit the metadata xml and remove the name for a framework reference the application crashes.

What is the expected result?
Validation / error message should be shown just like there is if you remove or mistype the targetFramework attribute

Screenshot of my example data:

For completeness find below my comments I wrongly put on an issue that I thought was the same one:
https://github.com/NuGetPackageExplorer/NuGetPackageExplorer/issues/702#issuecomment-502403154

when you manually edit the xml usually errors stop you from saving / continuing with bad data. For example if NuGet.Packaging.FrameworkReferenceGroup is missing its TargetFramework attribute you can't continue as the SaveMetadataAfterEditSource catches any exceptions thrown during the creation of EditablePackageMetadata.
However, NuGet.Packaging.FrameworkReferenceGroup has an IEnumerable of FrameworkReferences that are lazily evaluated due to LINQ. This means the usual mechanism of catching bad data doesn't kick in, then as soon as the WPF trys to bind the IEnumerable is evaluated and the exception is then thrown.
Weirdly, if you remove the name attribute from a package dependency you don't get an error (which you assume you would as its missing) instead the package entry is simply stripped from the collection. This points to inconsitencies in the NuGet.Packaging.Manifest class and how it handles bad data.
As this appears to be an issue in how the Manifest is parsed and also how its using lazy enumerations it's worth raising this in the NuGet repository. However is it worth trying to force the lazy evaluations as part of the SaveMetadataAfterEditSource method? It would feel a bit hacky, probably use the GetLastOrDefault() method Jon Skeet suggests, but it would ensure the app doesn't crash out on this or other issues of the same type.

https://github.com/NuGetPackageExplorer/NuGetPackageExplorer/issues/702#issuecomment-502403425

rather than sticking something like:
foreach (var group in source.FrameworkReferenceGroups) { _ = group.FrameworkReferences.LastOrDefault(); }
into the constructor of EditablePackageMetadata would it be more prudent to add a "Validate Package Source" method or an extension method on IPackageMetadata that can be appropriately called as part of SaveMetadataAfterEditSource?

Hi @onovotny , @mjjames.
I was able to recreate the issue when using NuGet.Packaging.dll 5.3.0-xprivate.60008.
I was not able to recreate the issue when using locally built NuGet.Packaging.dll 5.3.0-preview 2 from NuGet.Client repo. Editor throws validation error, see below:

I was able to recreate the issue when using NuGet.Packaging.dll 5.3.0-xprivate.60008.

I was not able to recreate the issue when using locally built NuGet.Packaging.dll 5.3.0-preview 2 from NuGet.Client repo. Editor throws validation error, see below:

I think we could close this one? (for now?) @onovotny what do you think?

Can we verify fixed in the latest (which uses 60011), based on later nuget code?

I can still reproduce the bug with NuGet.Packaging.dll version <IP_ADDRESS>011. (Which crashes the app if I continue debugging.)


Integration of Interventional Robustness Score

Interventional Robustness Score (IRS) implementation from the following paper:
Suter, R., Miladinović, Đ., Bauer, S., & Schölkopf, B. (2018).
Interventional Robustness of Deep Latent Variable Models.
arXiv preprint arXiv:1811.00007.
New implementation in Tensorflow off the original code

I have merged this change. Thanks for the contribution.


 It does not matter if tap-to-click is enabled or disable in Gnome Shell.
I'm unsure if this is a configuration error or caused by keyd's implementation.
$ cat /etc/keyd/internal.conf
[ids]
05ac:0262

[capslock]
h=left
j=down
k=up
l=right

[main]
capslock=overload(capslock, esc)
esc=capslock

$ uname -rsm
Linux 6.1.82 x86_64

$ keyd --version
keyd v2.4.3 ()

# lshw -numeric -sanitize -class input: lshw.txt
Maybe this issue is related?

Additional info:
Finger-taps with multiple fingers are interpreted/emitted as: kp1 down/up, kp6 down/up, kp8 up/down, kpplus up/down.

My issue seems to be related to https://github.com/rvaiya/keyd/issues/676 or even a duplicate.
The maintainer writes:

However, laptops will often group other keys into the touchpad device which can cause keyd to misidentify them as keyboards.

In my case, the keyboard and the touchpad have the same ID. So it seems that it is impossible to simply ignore the tap events by ignoring the relevant ID of the touchpad via -<id>.
journalctl -u keyd.service

Apr 12 00:02:03 titan keyd[1043]: Starting keyd v2.4.3 ()
Apr 12 00:02:03 titan keyd[1043]: DEVICE: ignoring 0fac:1ade  (keyd virtual pointer)
Apr 12 00:02:03 titan keyd[1043]: DEVICE: match    05ac:0262  /etc/keyd/internal.conf        (bcm5974)
Apr 12 00:02:03 titan keyd[1043]: DEVICE: match    05ac:0262  /etc/keyd/internal.conf        (Apple Inc. Apple Internal Keyboard / Trackpad)

As a temporary workaround, try this:
[id]
 *

it works on my mbp9,2.

My issue seems to be related to https://github.com/rvaiya/keyd/issues/676 or even a duplicate.

Indeed. I've added a note about this to the man page.

In my case, the keyboard and the touchpad have the same ID.

Try the latest commit, it should now allow you to distinguish between device notes with the same product and vendor id pairs. keyd monitor should produce unique ids for each pseudo device that gets created.

@rvaiya please excuse the delayed response.
I just got around to testing your commit (on a different Mac model mind you).
keyd indeed distinguishes between internal keyboard and mouse on Apple devices.
The internal keyboard is identified as:
Apple SPI Keyboard      0000:0000       v down
while the internal touchpad is identified as
Apple Inc. Magic Trackpad       05ac:0265       v up
Can I assume that the id of0000:0000` is to be expected here? Thanks!


Add test for support columns with different sizes as inputs to UDTFs

Fixes #176
Requires https://github.com/omnisci/omniscidb-internal/pull/5086

https://github.com/omnisci/omniscidb-internal/pull/5086 was merged


Add EE install instructions

We started changing Ubuntu instructions into a mix of Team and Enterprise Edition and it got too confusing, so reversing those and adding explicit Enterprise Edition install guides.

Merging these now to help a customer


Fix to/from closure type in JibExtension

fixes #1060

I don't know about Gradle, but logically, I feel like referencing an interface that has necessary getters and setters should have worked. Doesn't seem difference really, so not understanding why it didn't work.

@chanseokoh Gradle works fine, but the issue was IntelliJ wasn't able to pick up the interface annotations.

Oh, I see. Sounds like the fault of IntelliJ then.

The Target Image Params has extra params though, and it shouldn't be able to apply the action on it even if it wanted to?

@loosebazooka I think this is good after we fix the @Optional stuff. Can we get this in soon? I want to work on #1083 and I expect that to touch these files a bunch.

Yeah sorry, sometimes I forget I write code from time to time :disappointed:


[AVM Module Issue]: virtualmachine error when deploying vm with ZRS datadisk

Check for previous/existing GitHub issues

[x] I have checked for previous/existing GitHub issues

Issue Type?
Bug
Module Name
avm/res/compute/virtual-machine
(Optional) Module Version
0.10.0
Description
Hi,
Since module version 0.10.0 of virtualmachine we can't deploy a vm with one or more datadisks that will be joined to a availability zone.
The error we get is:
Status Message: A disk with sku 'Premium_ZRS' cannot be created in an availability zone. (Code:OperationNotAllowed)
We did a rollback to version 0.9.0 and deployments run successfull again.
(Optional) Correlation Id
No response

maybe a solution:
When storagetype (sku) is ZRS than set datadisk zone to 0. In the current version the datadisk zone follows the zone of the vm deployment. This is not possible.

@gerjon1990 thanks! Will look into it

Hey @rahalan,
Please triage this issue when you get the chance 🙂


[Dashboard] Snapshot Delegation Tool

Mel.eth is curious who's delegating to him - this could be a broad tool to measure who is delegating to whom and generally monitor our governance for individuals who are accumulating delegation power. This is also probably broadly useful across all snapshot governance systems, so could be useful for clout. Surprisingly, I don't think this has been built yet.
Delegations are on-chain, so this is queryable via Dune. This is doable for someone who's reasonably good at SQL, but requires a bit of work and thought.
Docs:
Snapshot resource here: https://docs.snapshot.org/guides/delegation
events documentation here: https://github.com/gnosis/delegate-registry/blob/main/contracts/DelegateRegistry.sol
Contract has been decoded on dune.xyz , e.g.
select *
from snapshot."DelegateRegistry_evt_SetDelegate"
limit 100;
Things to keep in mind:

Confirm that the space ID is actually being recorded in the event log. If the space ID is blank, then the delegation applies to all spaces.
Figure out how to map useful names of protocols to space IDs
You'll have to cancel out delegation events and delegation clearing events and figure out how to handle that. Some cases - what if someone doesn't clear their delegation and just re-delegates to someone else?
A delegation is for ALL of a user's tokens, so you'll need to join to current token balances in order to figure out what token to use for the voting power.

Hey @jackiep00, I know some devs who'd be interested in working on this. Is there a budget for fixing this issue, and if so, how much?
I'd put this issue up on OpenQ and list the budget publicly to attract devs.
Thanks!


flutter test can take more than a minute to execute a single test

reproduction steps
cd $FLUTTER/packages/flutter
flutter test --name "Floating action button directionality"

expected result
The test should take less than a minute to execute one test.
seen results
The test takes well over a minute and appears to be spending a lot of time "loading" tests that I'm not executing.
dart:frontend_server.dart.snapshot and dart:flutter_tools.snapshot are taking up a large portion of my CPU.
example output
aaclarke-macbookpro2:flutter aaclarke$ flutter test --name "Floating action button directionality"
01:09 +0: loading /Users/aaclarke/dev/flutter/packages/flutter/test/material/button_bar_theme_test.dart

doctor
[✓] Flutter (Channel unknown, 2.13.0-0.0.pre.537, on macOS 12.3.1 21E258 darwin-x64, locale en-US)
    • Flutter version 2.13.0-0.0.pre.537 at /Users/aaclarke/dev/flutter
    • Upstream repository unknown
    • Framework revision fd5356ff60 (9 days ago), 2022-04-13 12:09:11 -0700
    • Engine revision f6e7fd521c
    • Dart version 2.18.0 (build 2.18.0-18.0.dev)
    • DevTools version 2.12.2

Running by name has the unfortunate side effect that all tests must be invoked (which of course means they must be compiled) so that the test file containing said test can be located. if you narrow it down by giving it a single test file it should be much faster. i.e. flutter test -name foo path/to/test.dart

@gaaclarke Hi, Please try by passing single test file as suggested to see tests running faster.

@darshankawar, supplying the test file cuts down execution time.  I consider that a workaround and the bug is still a bug, albeit a lower priority bug since there is a workaround.

@gaaclarke Thanks for the feedback. Labeling it for further tracking.

@darshankawar, supplying the test file cuts down execution time. I consider that a workaround and the bug is still a bug, albeit a lower priority bug since there is a workaround.

@gaaclarke Note, you have to actually execute a file to resolve the full test name (which concatenates any surrounding group names with the final test() invocation).
I suppose you could try to compile them one by one and stop once you found it, which on average would mean compiling half of them; though I'm not sure if this method would take half the time.

I suppose you could try to compile them one by one and stop once you found it, which on average would mean compiling half of them; though I'm not sure if this method would take half the time.

Actually, you can have multiple tests with the same name, so you'd still need to compile all files

I suppose you could try to compile them one by one and stop once you found it, which on average would mean compiling half of them; though I'm not sure if this method would take half the time.

Actually, you can have multiple tests with the same name, so you'd still need to compile all files

Ohh yeah, I got bit by this!

I'm closing this as this behavior is implemented upstream in package:test https://github.com/dart-lang/test/issues


Custom models not working after relaunching shark

Using build shark_sd_20230204_496 of the windows exe I've had multiple safetensor models working by placing them in model folder.
Launching shark and selecting model in drop down, once i get the error about the init.py error and restarting shark
On the 2nd attempt I am able to generate images with the custom model.
close and relaunch shark i then run into this error below, i pasted the text file.
This is tested on the deliberate safe tensor model from Civitai, but appears to happen on all custom models so far and should be repeatable following the steps above

relaunchingsharkcustommodel.txt
Also here is a text file of error I get on the very first try of loading a custom model in shark, again closing shark then re-opening will work for that session but once we shut down shark we start running into the error above.
Firstattemptcustommodel.txt
EDIT: At the moment, fastest work around I found is to delete the clip file for the custom model and launch shark, it will regenerate the file and begin generating fairly quickly for now. for example for deliberate I delete the clip1_64_512_512_fp16_E_SharkSnap_models_deliberate_vulkan-00000000-0300-0000-0000-000000000000.vmfb file, then launch shark, select deliberate and generate, it takes a minute to re-create this file then start generating images.
windows 11
7900 xtx

Just curious if you used vae.pt along with safetensors file? dose vae enabled ?

Just curious if you used vae.pt along with safetensors file? dose vae enabled ?

Only thing I am currently doing in shark is downloading the safetensors file into the model folder of Shark, and selecting it in the drop down, if it fails on first try, relaunch shark and it should work on 2nd try. If you found that a VAE an be downloaded for certain models and improves generation please let me know or tell me how :)


contact section added

I Have added contact section
Please have a look
https://user-images.githubusercontent.com/84792579/207171420-3dfbdbd8-5e0d-40bb-92a1-58f1ab6bdf3f.mp4
It is showing "Message Sent Successfully"
I also have added the google live working map which is very helpful for this website
It also have all fields mandatory otherwise it gives message of required
Hope you like it
thank you!

I Have added contact section Please have a look
contact.mp4
It is showing "Message Sent Successfully" I also have added the google live working map which is very helpful for this website It also have all fields mandatory otherwise it gives message of required
Hope you like it thank you!

@Janhvi52 please review it if any change then please let me know

I Have added contact section Please have a look
contact.mp4
It is showing "Message Sent Successfully" I also have added the google live working map which is very helpful for this website It also have all fields mandatory otherwise it gives message of required
Hope you like it thank you!

i was telling about this "message sent successfully" thing riya and can you make some changes in the ui like the image in the background,its not as per the rest of the site and the footer section also, i've attached a ss you can follow this for patterns (not colors)

I Have added contact section Please have a look
contact.mp4
It is showing "Message Sent Successfully" I also have added the google live working map which is very helpful for this website It also have all fields mandatory otherwise it gives message of required
Hope you like it thank you!

 i was telling about this "message sent successfully" thing riya and can you make some changes in the ui like the image in the background,its not as per the rest of the site and the footer section also, i've attached a ss you can follow this for patterns (not colors)

hope you got me riya
thanks :)

I Have added contact section Please have a look
contact.mp4
It is showing "Message Sent Successfully" I also have added the google live working map which is very helpful for this website It also have all fields mandatory otherwise it gives message of required
Hope you like it thank you!

 i was telling about this "message sent successfully" thing riya and can you make some changes in the ui like the image in the background,its not as per the rest of the site and the footer section also, i've attached a ss you can follow this for patterns (not colors)

hope you got me riya thanks :)

If I remove that background image and make the background color black and done with that is not okay to be worked

@Janhvi52 can you please merge my this PR i am currently learning how to display that message so as I learned I can raise the new issue and adding more functionality of "submit successfully " Message to contact section till then I want to work on another section so can you please merge this one and assign new one

okay just revise the footer section once and refer the image above then will merge it thanks riya

@Janhvi52 can you please merge my this PR i am currently learning how to display that message so as I learned I can raise the new issue and adding more functionality of "submit successfully " Message to contact section till then I want to work on another section so can you please merge this one and assign new one

okay just revise the footer section once and refer the image above then will merge it thanks riya

Accordingly it is almost same I will try to improve it more in designing while making new PR it is okay with that @Janhvi52

@Janhvi52 can you please merge my this PR i am currently learning how to display that message so as I learned I can raise the new issue and adding more functionality of "submit successfully " Message to contact section till then I want to work on another section so can you please merge this one and assign new one

okay just revise the footer section once and refer the image above then will merge it thanks riya

Accordingly it is almost same #dark theme I will try to improve it more in designing while making new PR it is okay with that @Janhvi52

@Janhvi52 can you give me your review please

@Janhvi52 can you please merge my this PR i am currently learning how to display that message so as I learned I can raise the new issue and adding more functionality of "submit successfully " Message to contact section till then I want to work on another section so can you please merge this one and assign new one

okay just revise the footer section once and refer the image above then will merge it thanks riya

Accordingly it is almost same #dark theme I will try to improve it more in designing while making new PR it is okay with that @Janhvi52

@Janhvi52 can you give me your review please

okay make sure it meet the requirements properly in the next pr, I'm merging this!

@Janhvi52 can you please merge my this PR i am currently learning how to display that message so as I learned I can raise the new issue and adding more functionality of "submit successfully " Message to contact section till then I want to work on another section so can you please merge this one and assign new one

okay just revise the footer section once and refer the image above then will merge it thanks riya

Accordingly it is almost same #dark theme I will try to improve it more in designing while making new PR it is okay with that @Janhvi52

@Janhvi52 can you give me your review please

okay make sure it meet the requirements properly in the next pr, I'm merging this!

yes sure I am learning that one as i completed then i raise the new issue and done that task


[FLINK-21301][table-planner-blink] Decouple window aggregate allow lateness with state ttl configuration

What is the purpose of the change
The pr aims to decouple window aggregate allow lateness with state ttl configuration

Introduce table.exec.emit.allow-lateness configuration (which is also @Experimental).
Use the value of table.exec.emit.allow-lateness  as allow-lateness for window and also keep the state ttl can still take effect when the new config is not set table.exec.emit.allow-lateness.
Fix bug: enable allow-lateness for window only when late-fire emit strategy is specified

Brief change log

Updates WindowEmitStrategy

Verifying this change

Add UT in GroupWindowTest
Add ITCase in GroupWindowITCase

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

@flinkbot run azure

Looks good to me.


Unable to compile on ubuntu

Hello Sir,
Will be possible to send me PRG file to garmindescentmk1?
I have spend 5 hours trying to compile it on ubuntu but had no luck with VScode.
Much appriciated. Worked when ported fenix5plus prg file to it. but lost it unfortunately.
Best,
Tomas

My model is not awailable on the sotre


Excalidraw export png dark mode support.

When exporting the image in Excalidraw there is no way for it to export the png to show the dark mode variant.

can i work on this issue...

@burnthoney Actually you can toggle whether you want to enable or disable dark mode while exporting the png

Dark mode export currently doesn't work in Safari (and thus the "dark mode" toggle button isn't rendered there). We have a proof of concept fix for this, but it's not without downsides. Nevertheless we'll likely be releasing it soon, anyway, so stay tuned.


Get logged-in user stats from server

Now that we have a way for users to log in, stats for said users should be populated from the server's profile info:
## get stats
curl "https://api.5calls.org/v1/users/stats" \
     -H 'Cache-Control: max-age=0' \
     -H 'Authorization: Bearer <JWT>'

That'll get you something like this:
{
  "contact": 221,
  "voicemail": 158,
  "unavailable": 32
}

Being worked on in #209


link for the trained model

Hi, link in the checkpoint folder seems not accessible. Could you please share another one?

Make it after several times. Thank you.

Each time when it is to complete, the downloading is forbidden.

Hi Jiayi719,
It seems that I can still download it from the dropbox. But maybe you can try this link from onedrive, to see if it works.
https://1drv.ms/u/s!AthJr3PSm53WhN41CzRe0jYAZa7aPg?e=eaAWkM
Best,
Chenyang

Thank you. It works.


imp(ics24-host): missing storage paths

Details

The following paths are missing from the current set of ics24 paths (following ibc-go)

NextClientSequence
NextConnectionSequence
NextChannelSequence
ProcessedTimeKey
ProcessHeightKey

The last two come from Tendermint client implementations - but can be generalized for any host with timestamp and height concepts.
Version

<= v0.50.0

This is required to incorporate an AVL store in the current MockIbcStore as part of #677


fix(dsl): ambiguous alternatives

[log] [lsp] Ambiguous Alternatives Detected: <0, 1> in  inside <RelationFragment> Rule, <String, }, }> may appears as a prefix path in all these alternatives. See: https://chevrotain.io/docs/guide/resolving_grammar_errors.html#AMBIGUOUS_ALTERNATIVES For Further details.
[log] [lsp] Ambiguous Alternatives Detected: <0, 1> in  inside <RelationFragment> Rule, <String, IdTerminal, ->> may appears as a prefix path in all these alternatives. See: https://chevrotain.io/docs/guide/resolving_grammar_errors.html#AMBIGUOUS_ALTERNATIVES For Further details.

Fixed in #947


Creating candidate space fails for version 0.1.12

For the latest version of petab-select PEtab.jl errors. Running on Python 3.10 it boils down to:
select_problem = petab_select.Problem.from_yaml("petab_select_problem.yaml")
cs = petab_select.ui.candidates(problem=select_problem, criterion=select_problem.criterion)

Yielding the error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sebpe/anaconda3/envs/PeTab/lib/python3.10/site-packages/petab_select/ui.py", line 99, in candidates
    predecessor_model.get_criterion(
AttributeError: 'NoneType' object has no attribute 'get_criterion'

Everything worked prior to version 0.1.12 (noticed this once I managed to get PyCall workng on GitHub CI).
I used the petab-select problem file from here.

Thanks for reporting! This kind of error will be caught when the calibration tests are re-introduced soon.


[NullAway] dereferenced expression despite the explicit null check.

NullAway throws an error [NullAway] dereferenced expression allowedNames is @Nullable at  .filter(name -> allowedNames.contains(name)).  I was not expecting this error because of the explicit null check for allowedNames.
import java.util.stream.Collectors;
import java.util.Collections;
import java.util.List;
import java.util.Set;
import javax.annotation.Nullable;

 class TestStream{

    @Nullable private final Set<String> allowedNames;
    TestStream(@Nullable Set<String> allowedNames) {
      this.allowedNames = allowedNames;
    }

    private List<String> filterAllowedNames(List<String> inputNames){
      if (allowedNames == null) {
        return inputNames;
      }
      if (allowedNames.isEmpty()) {
        return Collections.emptyList();
      }
      return inputNames
              .stream()
              .filter(name -> allowedNames.contains(name))
              .collect(Collectors.toList());

    }
  }

Thanks for the report!  This case is very similar to that of #570.  The main difference is that in this case, the code is definitely safe since allowedNames is final, so it cannot be updated after the null check.  We should think about whether we'd want to handle this case separately from #570 since here we could prove safety even in cases not involving streams, whereas fixing #570 would involve heuristics.
@ketkarameya does this case come up frequently in your code base?

@msridhar This is less frequent scenario in the code base. I think this is a problem specifically when streams are used. NullAway does not report any error for the below program (Note that I have converted the stream pattern to for loop pattern).
class TestStream{

    @Nullable private final Set<String> allowedNames;
    TestStream(@Nullable Set<String> allowedNames) {
      this.allowedNames = allowedNames;
    }

    private List<String> filterAllowedNames(List<String> inputNames){
      if (allowedNames == null) {
        return inputNames;
      }
      if (allowedNames.isEmpty()) {
        return Collections.emptyList();
      }
      List<String> output = new ArrayList<>();
      for (String name : inputNames){
        if (allowedNames.contains(name)){
          output.add(name);
        }
      }
      return output;
    }
  }

Thanks!  Yup, the issue with the streams code is that allowedNames is accessed from a lambda, and in general NullAway has no idea when that lambda will run (e.g., it could get stored in a field and run much later).  For a non-final field, the field might get set to null after the null check but before the lambda executes.  In your case, the field is final so this is not possible.  Further, for streams, we could bake in some knowledge that the callbacks run "immediately" (before the enclosing function returns) and hence we don't need to worry about the field being nulled out in between.

Note that the following workaround would be sufficient for NullAway to know that the code is safe (though, in the case with final, I agree this shouldn't be needed):
    private List<String> filterAllowedNames(List<String> inputNames){
      if (allowedNames == null) {
        return inputNames;
      }
      if (allowedNames.isEmpty()) {
        return Collections.emptyList();
      }
      return inputNames
              .stream()
              .filter(name -> allowedNames != null && allowedNames.contains(name))
              .collect(Collectors.toList());

    }

Playing devil's advocate - Adding an additional null check in the filter (when I am already sure allowedNames is not null) can introduce performance overhead (e.g.  when inputNames contains 10s of thousands of entries).


Connection Pool Errors

Hello,
I've been using this library with poolboy in production for a couple of days and just received this error. I'm not sure what my next step should be. My first thought was to increase the timeout or pool_timeout (note this app isn't under heavy load, 1-2 users at max).
Should I remove poolboy and make the connection for each query? I'm mainly using it on every GraphQL request to fetch the user and add it as context for every request so it'll be used quite often.
Elixir Version: 1.3.2
MongoDB Version: 2.6.x (SSL not being used)
Host: Heroku for the app and Compose.io for MongoDB
(using Phoenix 1.2)
Any idea of what I should do next?
2017-02-01T01:41:50.691003+00:00 app[web.1]: 01:41:50.690 [error] Mongo.Protocol (#PID<0.333.0>) disconnected: ** (Mongo.Error) tcp recv: unknown POSIX error - :timeout
2017-02-01T01:41:50.692470+00:00 app[web.1]: 01:41:50.690 [error] Mongo.Protocol (#PID<0.337.0>) disconnected: ** (Mongo.Error) tcp recv: unknown POSIX error - :timeout
2017-02-01T01:41:50.692471+00:00 app[web.1]: 01:41:50.690 [error] Mongo.Protocol (#PID<0.335.0>) disconnected: ** (Mongo.Error) tcp recv: unknown POSIX error - :timeout
2017-02-01T01:41:50.723329+00:00 app[web.1]: 01:41:50.723 [error] GenServer #PID<0.333.0> terminating
2017-02-01T01:41:50.723332+00:00 app[web.1]: ** (RuntimeError) disconnect/2 not implemented
2017-02-01T01:41:50.723334+00:00 app[web.1]:     (mongodb) lib/db_connection.ex:240: Mongo.Protocol.disconnect/2
2017-02-01T01:41:50.723335+00:00 app[web.1]:     (db_connection) lib/db_connection/connection.ex:178: DBConnection.Connection.disconnect/2
2017-02-01T01:41:50.723335+00:00 app[web.1]:     (connection) lib/connection.ex:767: Connection.disconnect/3
2017-02-01T01:41:50.723335+00:00 app[web.1]:     (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
2017-02-01T01:41:50.723336+00:00 app[web.1]:     (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
2017-02-01T01:41:50.723336+00:00 app[web.1]:     (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
2017-02-01T01:41:50.723914+00:00 app[web.1]: 01:41:50.723 [error] GenServer #PID<0.337.0> terminating
2017-02-01T01:41:50.723915+00:00 app[web.1]: ** (RuntimeError) disconnect/2 not implemented
2017-02-01T01:41:50.723916+00:00 app[web.1]:     (mongodb) lib/db_connection.ex:240: Mongo.Protocol.disconnect/2
2017-02-01T01:41:50.723917+00:00 app[web.1]:     (db_connection) lib/db_connection/connection.ex:178: DBConnection.Connection.disconnect/2
2017-02-01T01:41:50.723918+00:00 app[web.1]:     (connection) lib/connection.ex:767: Connection.disconnect/3
2017-02-01T01:41:50.723918+00:00 app[web.1]:     (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
2017-02-01T01:41:50.723919+00:00 app[web.1]:     (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
2017-02-01T01:41:50.723920+00:00 app[web.1]:     (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

There's an open PR for this that has some work left to do: https://github.com/ericmj/mongodb/pull/99.

@ericmj oh great thanks! Would you recommend not using Poolboy until that PR is closed? Or is this error something that won't affect the app in production?

If the connection goes down, in some circumstances, the connection will not be automatically reconnected. Instead the connection process goes down, the supervisor will restart the process and the reconnection will happen. If your database is down for a while this has a chance of killing your supervisor depending on your supervisor settings.

I believe this has been fixed in #116.

@ankhers
I am using mongodb from an Elixir app and I am using - {:mongodb, "== 0.4.6"}
but I still see this issue, can you please help!
Mongo.Protocol (#PID<0.25313.3>) disconnected: ** (Mongo.Error) tcp recv: unknown POSIX error - :timeout

@nitinjain1105 Can you please create a new issue?

@ankhers
Created new issue#274


Investigate viability of EEBO-TCP integration

@rlskoeser Here is a working list of titles we are interested in adding (our undergraduate intern is actively working on this spreadsheet, FYI!). The spreadsheet includes URLs to the items and has a separate column for the volume IDs, which I took from the URL bar of the EEBO-TCP site. There are some sample excerpts in the spreadsheet, too: the pages are strange, however. Most of the pages are "unnumbered" so I took the numbers that change on the page URL following the ID, which seem to indicate the full section.
We're interested at this point in getting a sense of what the data structure is like and how easy or hard it might be to integrate it into the existing PPA structure (for instance, is the unnumbered page numbers a big blocker? is there metadata for those somewhere else that I'm not seeing?). Are image thumbnails impossible to include since they don't seem to be on the TCP site? How would we pull the metadata? Etc.

@mnaydan I started looking into this last week and should have added some notes while it was fresh.
The xml structure doesn't look too bad; I think we'll have to write some custom parsing code but it doesn't need to be very complicated since we mainly want pages, page numbers, and plain text.  (This is based on looking at the collection that's published on GitHub; I'm assuming they are similar.). I don't think unnumbered pages is a blocker, we'd just want a way to handle pages with no label.
I think the metadata would be similar to what we did for the Gale/ECCO records, where we'd have to rely on MARC records already purchased by PUL. Fortunately they provide a mapping for us: https://textcreationpartnership.org/using-tcp-content/eebo-tcp-cataloging-records/
I am wondering if there's a way for us to use PUL's [unadvertised] bib API to get MARC records instead of having to store a local copy that we can lookup as needed (as we did for ECCO). That would depend on whether we can look it up by the id we'll need to use and if the API is fast enough (assuming we're allowed to use for this).  I expect PUL owns the MARC records, but we should confirm.
I was wondering how we get the full text, and I found in the faq that there are some bulk downloads; I'm not certain if this is what we want or not: https://textcreationpartnership.org/faq/#faq05   (There's a collection published on GitHub, but it isn't all of the texts).
They do have a list of projects using the content; we can ask them to add PPA when/if we import content: https://textcreationpartnership.org/using-tcp-content/projects-and-publications-using-tcp-texts/
I don't see any mention of images anywhere; getting access to images for thumbnails might require negotating with ProQuest (if it's even possible).
We might want to be in contact with the TCP folks at UMich to let them know what we're working on, so we could ask them for advice on how we might get thumbnail images.

After looking back at our ECCO metadata documentation, I emailed Joe Marciniak at PUL to request access to the EEBO MARC records via a local stored copy, XML format preferred (we decided it would be smart to repurpose the code we used for ECCO as much as possible, rather than going the API route).
I also emailed the TCP folks at<EMAIL_ADDRESS>to ask them about the full-text and thumbnails.

@mnaydan I downloaded the bulk exports and did a quick check against the ids in the spreadsheet you shared  and the id lists in the text files they provide - all of our IDs are present, and the majority of them are included in phase 1, only three of them are in the phase 2 set.

Investigation task is done - should be able to integrate

use existing code for MARC records
will need some lightweight xml parsing to get the text we care about.