DecentralML -  grant application

Project Abstract
One-liner: A Polkadot protocol for decentralised federated machine learning and collective governance.
Project Description: DecentralML introduces decentralised federated machine learning (DFML), governed by collective consensus to the Polkadot ecosystem. Our goal is to provide a robust framework for AI model developers, organisations, and applications, enabling decentralised ownership of models while ensuring privacy and scalability. With node or on-device training protecting privacy, the ability to share data training between organisations (nodes), collaborative AI training, and "collective" governance controls, DecentralML may transform the field of machine learning for state-of-the-art AI models (think LLMs and more) with transparent governance.
This is a new funding application and is not in response to an RFP, nor a follow-up grant.
Grant level

[ ] Level 1:  Up to $10,000, 2 approvals
[x] Level 2:  Up to $30,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[yes] The application template has been copied and aptly renamed (project_name.md).
[ yes] I have read the application guidelines.
[ yes] Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
[ yes] The software delivered for this grant will be released under an open-source license specified in the application.
[ yes] The initial PR contains only one commit (squash and force-push if needed).
[ yes] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
[ ] I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks a lot for the application. The Federated Learning Consensus especially sounds interesting. Do you know about the initiative by the co-founder Alex Skidanov of NEAR? See for example https://www.youtube.com/watch?v=2KM9N3jFdSk Regarding the deliveries, could you provide as many details as possible (functionality, programming language, etc.) and exclude deliveries that you are not sure how to implement these, like "Data Management." The reason is that we need to have as many details as possible. The milestone tables are the requirements of our contracts.

@Noc2 Thank you, we will check out the NEAR initiative and flesh out the deliverables a little more. Please can you tell us what level of detail we need and the deadlines that might be involved to get into the "Q2 batch"?

@Noc2 Thank you, we will check out the NEAR initiative and flesh out the deliverables a little more. Please can you tell us what level of detail we need and the deadlines that might be involved to get into the "Q2 batch"?

We don't have any deadlines and accept grants on a continuous basis. Regarding the details, the more, the better ;-)

We don't have any deadlines and accept grants on a continuous basis. Regarding the details, the more, the better ;-)

Hey @Noc2,
How are you? Hope all is well. We've updated our application based on thorough research we've conducted over the past couple weeks. We've decided to put aside the proxyModel approach for now due to the widespread use and commercial support of TensorFlow's Federated Machine Learning. We're adopting TensorFlow's approach, which allows any data scientist to transform a regular model into a federated one. Then by using DecentralML, they can take advantage of on-chain incentives and transparency. We're excited to hear your thoughts. Thanks a lot!

Excellent work, thank you @takahser !  The fixes for formatting have been committed.
We are keen to get started and plan our next months, if you, @Noc2 or the team need anything please let us know.
Many thanks

@takahser thank you for the great questions.

what are the trust assumptions between the participants that train a common model? For example, in the link: https://www.nature.com/articles/s41598-022-12833-x federated learning paper for private medical data you shared the assumption is that the participating clinics trust each other.

Answer: In our approach, we're using the Tensorflow Federated Learning single entity implementation, which assumes that the model creator is the only trusted party. Unlike the federated entity approach, our method doesn't require mutual trust between all participants. While we might consider implementing the a decentralized multi-party approach in the future, it would necessitate substantial modifications to the Tensorflow FL library.

What's the incentives for Data Attributers to act honestly and not mislabel the data with some arbitrary tag?

Answer: Ensuring honesty among Data Attributers can be a complex task. Initially, we'll implement a basic game theory consensus to this challenge. We plan to use the Behavioural Strategy (https://en.wikipedia.org/wiki/Strategy_pattern) software pattern, which allows us to switch implementation based on parameters passed by the Model Creator when a model is initially uploaded. This will allow us to adapt and refine the strategy according to the model objectives.

You're mentioning token staking economy - do you mean tokenomics by that?

Answer: Yes, by token staking economy, we're referring to DecentralML’s tokenomics. When a model is uploaded, a certain amount of token is staked and divided currently into three pools for governance: Data Contributors, Model Engineers, and Data Annotators. This staked amount, paid by the Model Creator, is stored on-chain and used to incentivize these three groups. We also set the percentage of the pool allocated to each group, as well as the charges for downloading the model.

In general, what are the incentives for using federated learning?

Answer: Users can benefit from earning a portion of the staked tokens and maintain control over their models. Additionally, the use of the model itself can also serve as an incentive such as for the model creator.

In general, what kind of ML-based applications would you envision your platform to be used/what kind of models would be trained?

Answer: Great question, DecentralML leveraging TensorFlow Federated Learning, can support a wide range of applications. Typically these models are best generated from crowd source data sources.  Such as G-board Google’s keyboard which predicts the next word based on gradients sent by millions of devices. The versatility and popularity of TensorFlow make it a good choice for DecentralML as a large of number of machine learning engineers use it. We anticipate this will help drive adoption and transformation in how we begin to transparently train and govern models.

Answer: In our approach, we're using the Tensorflow Federated Learning single entity implementation, which assumes that the model creator is the only trusted party. Unlike the federated entity approach, our method doesn't require mutual trust between all participants. While we might consider implementing the a decentralized multi-party approach in the future, it would necessitate substantial modifications to the Tensorflow FL library.

A) If you're not using a "decentralized multi-party approach" I'm not sure what the benefits are of doing this on-chain. Why not just use one of the centralised systems instead?

Answer: Ensuring honesty among Data Attributers can be a complex task. Initially, we'll implement a basic game theory consensus to this challenge. We plan to use the Behavioural Strategy (https://en.wikipedia.org/wiki/Strategy_pattern) software pattern, which allows us to switch implementation based on parameters passed by the Model Creator when a model is initially uploaded. This will allow us to adapt and refine the strategy according to the model objectives.

B1) So in other words, the "game theory consensus" will be configurable by using the behavioural strategy?
B2) What kind of game theory would you use though? Could you elaborate on how that would look like?

Answer: Users can benefit from earning a portion of the staked tokens and maintain control over their models. Additionally, the use of the model itself can also serve as an incentive such as for the model creator.

C1) By Users do you refer to Data Contributors, Model Engineers, and Data Annotators? Or do you mean users that consume the model?
C2) Also, is there any mechanism to increase the amount of tokens in the 3 pools or is it a finite amount that the model creator pays and once the pools are drained there'd be no more incentive for the 3 groups to work on it?
C3) I'm confused that you're mentioning data contributors and how they're incentivised here, while earlier you stated that there's going to be only 1 model creator. Or do you mean by that that there is 1 model creator that doesn't bring any data, but there can be multiple data contributors? If the latter is the case, isn't it the same mechanism as described in the linked paper. Hence, it'd raise the question on how the data feed provider and their data can be trusted.

Answer: Great question, DecentralML leveraging TensorFlow Federated Learning, can support a wide range of applications. Typically these models are best generated from crowd source data sources. Such as G-board Google’s keyboard which predicts the next word based on gradients sent by millions of devices. The versatility and popularity of TensorFlow make it a good choice for DecentralML as a large of number of machine learning engineers use it. We anticipate this will help drive adoption and transformation in how we begin to transparently train and govern models.

You're mentioning G-board, an AI-enhanced keyboard here. Can you estimate how many devices you'd need for the model to become useful here? If it's in the millions (as you mention in the example), do you have a plan on how to onboard them and convince them to use your platform?

From: S E R A Y A @.>
Sent: 02 August 2023 22:24
To: w3f/Grants-Program @.>
Cc: Ashley Turing @.>; Author @.>
Subject: Re: [w3f/Grants-Program] DecentralML - grant application (PR #1818)
Answer: In our approach, we're using the Tensorflow Federated Learning single entity implementation, which assumes that the model creator is the only trusted party. Unlike the federated entity approach, our method doesn't require mutual trust between all participants. While we might consider implementing the a decentralized multi-party approach in the future, it would necessitate substantial modifications to the Tensorflow FL library.

If you're not using a "decentralized multi-party approach" I'm not sure what the benefits are of doing this on-chain. Why not just use one of the centralised systems instead?

In terms of querying the merits of DecentralML in comparison to unspecified centralised systems we like to remember the core essence of decentralisation is to provide levels of transparency, inclusivity, and accessibility that centralised systems inherently lack. With a 2-month timeline, we're not just building a project, but a cornerstone for TensorFlow developers, which are a huge segment of the machine learning community! This allows them to augment their existing work with the power of Polkadot's decentralised technology. Recognising the value this brings to the Polkadot ecosystem, we're somewhat taken aback by your question. But to clarify any confusion here are high-level immediate benefits delivered by this grant proposal in addition to adding substrate support option for all Tensorflow developers:
a. Data Annotators: As a framework it opens the door to a diverse array of dApps, obviously an opportunity that a centralised solution cannot provide. This not only expands the Polkadot ecosystem but also paves the way for varied data annotation methods which is hugely valuable for ML in general.
b. Model Engineers: Decentralisation fosters an inclusive platform for a broad range of participants. It ushers in a new era of on-chain governance, enabling democratic decision-making processes far beyond the scope of centralised solutions.
c. Data Contributors: For the first time, contributors can use on-chain undeletable accountability, transparency and be rewarded.
d. Clients: Decentralisation encourages competition, innovation, and diversity of models available for commercial, contribution, or educational purposes. This leads to superior models and a thriving marketplace.
DecentralML is not just a project; it's a ground-breaking step towards an inclusive, transparent, and robust AI ecosystem. We're laying the groundwork for a system that could support a multi-party approach, promoting collaboration and shared learning among a wider network of participants. Let's not delay and start this journey to revolutionise the AI landscape, we suggest we have a call if these points aren’t clear to you.
Answer: Ensuring honesty among Data Attributers can be a complex task. Initially, we'll implement a basic game theory consensus to this challenge. We plan to use the Behavioural Strategy (https://en.wikipedia.org/wiki/Strategy_pattern) software pattern, which allows us to switch implementation based on parameters passed by the Model Creator when a model is initially uploaded. This will allow us to adapt and refine the strategy according to the model objectives.
B1) So in other words, the "game theory consensus" will be configurable by using the behavioural strategy?
Yes
B2) What kind of game theory would you use though? Could you elaborate on how that would look like?
The premise is to incentivise high-quality contributions in a trustless environment. Here's a high level how we plan to implement it: Assign a base reward to each Data Annotator for their participation. This encourages all Annotators to contribute. Next, we introduce a validation mechanism. In each round, a subset of annotations made by each Annotator are “gamed”/ selected by the system and presented to other Annotators for validation. This obviulsy needs to account for the possibility of collusion, so ensure that the identity of the original Annotator is hidden during this process. Validators are then assigned different trust levels based on the level of consensus reached on their annotations (and perhaps a separate extensible rank algorithm). Essentially, the quality of each Annotator's work is assessed trustlessly, based on the degree of agreement among validators. Annotators whose work garners a higher level of agreement are assumed to provide more accurate and high-quality labels, and they receive a larger share of the reward pool. Conversely, those with lower agreement levels receive a smaller share. This happens iteratively, creating a dynamic and self-regulating system. This approach leverages the principles of game theory to foster and reward quality and accuracy.
Answer: Users can benefit from earning a portion of the staked tokens and maintain control over their models. Additionally, the use of the model itself can also serve as an incentive such as for the model creator.
C1) By Users do you refer to Data Contributors, Model Engineers, and Data Annotators?  Or do you mean users that consume the model?
Data Annotators, Model Engineers, Data Contributors AND Clients
C2) Also, is there any mechanism to increase the amount of tokens in the 3 pools or is it a finite amount that the model creator pays and once the pools are drained there'd be no more incentive for the 3 groups to work on it?
Yes and it also takes a percentage of whatever is remaining and given there 9+ decimal places this could go for quite a lot.
C3) I'm confused that you're mentioning data contributors and how they're incentivised here, while earlier you stated that there's going to be only 1 model creator. Or do you mean by that that there is 1 model creator that doesn't bring any data, but there can be multiple data contributors? If the latter is the case, isn't it the same mechanism as described in the linked paperhttps://www.nature.com/articles/s41598-022-12833-x. Hence, it'd raise the question on how the data feed provider and their data can be trusted.
Answer: Great question, DecentralML leveraging TensorFlow Federated Learning, can support a wide range of applications. Typically these models are best generated from crowd source data sources. Such as G-board Google’s keyboard which predicts the next word based on gradients sent by millions of devices. The versatility and popularity of TensorFlow make it a good choice for DecentralML as a large of number of machine learning engineers use it. We anticipate this will help drive adoption and transformation in how we begin to transparently train and govern models.
You're mentioning G-board, an AI-enhanced keyboard here. Can you estimate how many devices you'd need for the model to become useful here? If it's in the millions (as you mention in the example), do you have a plan on how to onboard them and convince them to use your platform?
—
Reply to this email directly, view it on GitHubhttps://github.com/w3f/Grants-Program/pull/1818#issuecomment-1662982600, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGXTVY7S6EABD4PSE62MQTDXTLAP7ANCNFSM6AAAAAAZQR7XY4.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

A) If you're not using a "decentralized multi-party approach" I'm not sure what the benefits are of doing this on-chain. Why not just use one of the centralised systems instead?

Firstly, federated machine is multi-party: the model will be trained with a “decentralized multi-party approach” by each user training their own local model, and the model aggregating the weights of all the local models. This doesn’t require the central model to trust each model, as the gradients can be verified with cross validation. Furthermore, in terms of querying the merits of DecentralML in comparison to unspecified centralised systems just like to point out the obvious: Decentralisation provides levels of transparency, inclusivity, and accessibility that centralised systems inherently lack. Within the proposed 2-month timeline, we're not just building a project, but a cornerstone for TensorFlow developers, which are a huge segment of the machine learning community. This allows them to augment their existing work with the power of tokenisation/ governance Polkadot's decentralised technology. Recognising the value this brings to the Polkadot ecosystem, we're somewhat taken aback by your question. But to clarify any confusion here are high-level immediate benefits delivered by this grant proposal in addition to adding substrate support option for all Tensorflow developers:
a. Data Annotators: As a framework it opens the door to a diverse array of dApps, obviously an opportunity that a centralised solution cannot provide. Also, brings in transparency into training. This not only expands the Polkadot ecosystem but also paves the way for varied data annotation methods which is hugely valuable for ML in general.
b. Model Engineers: Decentralisation fosters an inclusive platform for a broad range of participants. It ushers in a new era of on-chain governance, enabling democratic decision-making processes far beyond the scope of centralised solutions.
c. Data Contributors: For the first time, contributors can use on-chain undeletable accountability, transparency and be rewarded.
d. Clients: Decentralisation encourages competition, innovation, and diversity of models available for commercial, contribution, or educational purposes. This leads to superior models and a thriving marketplace.
We see DecentralML as an important step towards an inclusive, transparent, and robust AI ecosystem. We're laying the groundwork promoting transparency, collaboration and shared learning among a wider network of participants.

B1) So in other words, the "game theory consensus" will be configurable by using the behavioural strategy?

Yes that's the plan.

B2) What kind of game theory would you use though? Could you elaborate on how that would look like?

The premise is to incentivise high-quality contributions in a trustless environment. Here's a high level how we plan to implement it: Assign a base reward to each Data Annotator for their participation. This encourages all Annotators to contribute. Next, we introduce a validation mechanism. In each round, a subset of annotations made by each Annotator are “gamed”/ selected by the system and presented to other Annotators for validation. This obviously needs to account for the possibility of collusion, so ensure that the identity of the original Annotator is hidden during this process. Validators are then assigned different trust levels based on the level of consensus reached on their annotations (and perhaps a separate extensible rank algorithm). Essentially, the quality of each Annotator's work is assessed trustlessly, based on the degree of agreement among validators. Annotators whose work garners a higher level of agreement are assumed to provide more accurate and high-quality labels, and they receive a larger share of the reward pool. Conversely, those with lower agreement levels receive a smaller share. This happens iteratively, creating a dynamic and self-regulating system. This approach leverages the principles of game theory to foster and reward quality and accuracy.

C1) By Users do you refer to Data Contributors, Model Engineers, and Data Annotators? Or do you mean users that consume the model?

Data Annotators, Model Engineers, Data Contributors AND Clients

C2) Also, is there any mechanism to increase the amount of tokens in the 3 pools or is it a finite amount that the model creator pays and once the pools are drained there'd be no more incentive for the 3 groups to work on it?

Yes and it also takes a percentage of whatever is remaining (given there several decimal places this could go for quite a lot).

C3) I'm confused that you're mentioning data contributors and how they're incentivised here, while earlier you stated that there's going to be only 1 model creator. Or do you mean by that that there is 1 model creator that doesn't bring any data, but there can be multiple data contributors? If the latter is the case, isn't it the same mechanism as described in the linked paper. Hence, it'd raise the question on how the data feed provider and their data can be trusted.

To clarify: Data Annotators, Data Contributors, Model Engineers earn a portion of the staked tokens, with Model Engineers having collective governance and control over the model. Clients benefit by the actual use of the model itself.

You're mentioning G-board, an AI-enhanced keyboard here. Can you estimate how many devices you'd need for the model to become useful here? If it's in the millions (as you mention in the example), do you have a plan on how to onboard them and convince them to use your platform?

The number of Data Contributors depends on largely on the problem being solved and which parts of the DecentralML framework the model creator would like to employ. DecentralML is a framework that provides a solution to many businesses, brands, individuals etc. that are looking to leverage Tensorflow FL decentralised governance and incentivation. There might, perhaps, be some general understanding of the Tensorflow / ML / Federated Machine learning that you might be missing, as there are many ways DecentralML / Tensorflow could be used.
The number of Data Contributors varies depending on the specific problem being solved and the parts of the DecentralML framework the model creator chooses to use. The goal with DecentralML is to offer a versatile solution that caters to various businesses, brands, and individuals seeking to leverage Tensorflow FL decentralized governance and incentivization. There might be some gaps in your understanding of Tensorflow/ML/Federated Machine Learning, don't worry. There are numerous possibilities for utilizing DecentralML/Tensorflow FL.

Thank you for your detailed feedback and insights @takahser . We genuinely appreciate the depth of your analysis (reading the associated papers) and the critical points you've raised. Addressing your concerns:

Regarding the Institutions and Their Models:

You've rightly observed that in the papers we cited, institutions are known to each other and have their reputation at stake. This indeed acts as a natural disincentive against fraudulent activities. While medical models might employ this technique, our focus is broader. We aim to apply the principles of machine learning and federated learning to a range of problems, thus the coupling with Tensorflow. In any case, the nuances of each application might vary, and we believe that a comprehensive discussion on this might divert from the core objective of our grant application as a versatile framework.

Data Poisoning Attacks & Validation Concerns:

We understand the importance of ensuring data integrity and preventing malicious activities. Our reliance on Tensorflow's federated learning library is precisely because of its robustness and the extensive investment in its development. The library offers mechanisms to validate data submissions notably the TensorFlow Lite client. Furthermore, each gradient or weight upload can be cross-validated against a secret test dataset provided by the model uploader. Only those uploads that enhance the model's accuracy would then be integrated into the master model.

Annotators' Coordination & Game Theory:

Your point about the potential for annotators to collude is valid. The exact mechanism to prevent such collusion would indeed depend on the specific model and the available metadata about the annotators. Our intention to use the Strategy pattern is to allow flexibility in addressing such challenges. As we progress into the implementation, we'll delve deeper into research, like this paper, to address these concerns. However, it is important to note, DecentralML, as an extension to TensorFlow FL, aims to provide a framework that incentivises not just Data Annotators but also Model Engineers, Data Contributors, and Clients. Our goal is to offer a versatile platform that caters to a wide range of needs not just Data Annotators, we would like to stress the overall value this brings to the Polkadot.
In conclusion, we're committed to addressing the challenges you've highlighted and ensuring that DecentralML stands as a reliable and robust platform. We're eager to move forward with the implementation phase, where many of these concerns will be addressed in detail, and of course we expect this scrutiny when we submit our milestones. Your expertise and feedback will be invaluable to us making this a significant contribution, and we look forward to your continued guidance as we advance this grant application.

In any case, the nuances of each application might vary, and we believe that a comprehensive discussion on this might divert from the core objective of our grant application as a versatile framework.

I tend to disagree on this. In my opinion, the game theory behind a project like this is crucial to its success of a project. If this challenge is not solved, I don't see any sense supporting it here, to be honest with you.

Furthermore, each gradient or weight upload can be cross-validated against a secret test dataset provided by the model uploader. Only those uploads that enhance the model's accuracy would then be integrated into the master model.

That's good information. Could you lead me to some resources that describe or confirm how well that already works in the current version of Tensorflow?

Furthermore, each gradient or weight upload can be cross-validated against a secret test dataset provided by the model uploader. Only those uploads that enhance the model's accuracy would then be integrated into the master model.

Usually we'd expect our grantees to have a detailed plan of what they're trying to implement. While some challenges can not be avoided during development, in your case it seems to be a challenge quite central to your project. Hence, there's a rather high risk associated to it because the conclusion could potentially be that you're not able to solve them.
Regarding the paper you've mentioned, could you elaborate on its relevance to these challenges? Note that I haven't taken a look at it yet.
For now, I'm not going to accept it. Maybe if you reduce the scope to the parts of the project that are solved I'd change my mind, though I don't have a specific recommendation of how this constellation would look like. I'll mark it ready for review, so other members of the W3F grants committee can have a look as well - they might have a different opinion on it.

@takahser thanks - I will discuss with the others and respond to this more fully early next week, however, just to clarify one point here:

In any case, the nuances of each application might vary, and we believe that a comprehensive discussion on this might divert from the core objective of our grant application as a versatile framework.

I tend to disagree on this. In my opinion, the game theory behind a project like this is crucial to its success of a project. If this challenge is not solved, I don't see any sense supporting it here, to be honest with you.

It appears there's been a misunderstanding. Our reference was to the significance of discussing various Federated Learning approaches, particularly those outside of Tensorflow. The application domains highlighted in the cited papers you summarised aren't directly supported by Tensorflow so we didnt see a point going much further through them in this grant application. This wasn't in relation to the "game theory" of Data Annotators, it appears there might be some gaps in your understanding of Tensorflow. Additionally, it seems you might have missed our consistent emphasis on the adaptability of our approach, facilitated by the Strategy pattern. When an individual, company, or entity has built trust with Data Annotators outside of the DecentralML framework, the intricate consensus game theory, which seems to be your primary concern, becomes less critical. However our approach provides adaptable to accommodate "game theory" (which is more fully detailed, as you requested, in the paper provided,).
Hope this helps clarify.
Have a good weekend.

pinging @livetreetech

@semuelle @takahser
Hope this finds you well. Apologies for the delay, I am on paternity leave and have just managed to get some responses from the team regarding your comments and this application.
We understand your concerns regarding validation and the "game theory" aspect. The paper (https://arxiv.org/pdf/2101.03300.pdf) delves into validation techniques in a trustless environment. We are confident that a similar approach in combination with cross-validation techniques, which are foundational to ML will more than satisfy your main “game-theory” point of concern. Techniques like k-fold cross-validation and stratified sampling ensure the integrity and accuracy of model training. These methods, combined with our framework, address the "game theory" concerns and ensure a robust machine learning across incorporated Validation and Test data sets.
Our primary goal is to enhance the ML workflow and integrate core Tensorflow library functionalities with Substrate, which we believe could be invaluable to the Polkadot ecosystem and add a new paradigm to ML. We're committed to addressing the challenges you’ve raised and delivering a high-quality product. We are open to constructive feedback and look forward to progressing the application and working collaboratively to achieve our shared objectives.  Naturally, we accept that this is a milestone based grant so if we do not produce our milestones to your team’s requisite standard we will not be compensated.
Given this, we would ask that if you do not wish to support this application for reasons outside of this application then you please let us know. We feel we have a highly knowledgable and experienced team whom have addressed your concerns, if you simply do not want to support the application, or have concerns outside the scope of this application that might influence the decision, we'd appreciate transparency to ensure a productive dialogue.  Hope this answers all of your concerns without further time needed to schedule a call with all the various parties, however, if you feel it is then do let us know.  Thanks and have a nice weekend.

Thank you @takahser  for your detailed feedback and for recognising the value of a collaborative approach in grant evaluations. We appreciate the time you've invested in understanding the nuances of our proposal.
You've raised an excellent point regarding the random role assignment in VBFL and its implications for game theory and mechanism design. In VBFL, the dynamic role assignment indeed serves as a mechanism to incentivise good behavior among participants. Our initial design, which assigns fixed roles to Model Engineers, Data Annotators, Data Contributors, and Clients, stems from a more traditional ML workflow perspective.
However, I will discuss it further with the team and we will definitely consider incorporating a similar random role assignment mechanism into our framework. Specifically:
The "machine learning worker" in VBFL could correspond to our Data Contributor role, which is primarily managed by the TensorFlow Lite client. We will investigate the feasibility of integrating this functionality into our framework, mindful of any potential dependency issues and the fact that the "node" might not have any useful correctly formatted data to capture.
The roles of "model validator" and "blockchain miner" in VBFL align closely with the validation processes we've been discussing. While our framework focuses on validating both Data Contributors and Data Annotators, your suggestion to include dynamic role-switching, in addition to validation techniques like k-fold cross-validation and stratified sampling would indeed enhance the integrity and robustness of our framework, so thank you for the input.  Of course, the devil is always in the detail and when we get into implementation we will be working through the suggestion. Anyhow, incorporating these elements would address the "game theory" concerns you've highlighted and would make our framework even more robust and adaptable.
Given these clarifications and our willingness to adapt our approach based on your valuable feedback, we kindly request that you consider approving our grant application so we can move forward with implementation.
We look forward to your positive response and are available for further discussions, including a scheduled call if you deem it necessary.

Understood thank you for the clarification. Please can we now make plans to get started or do we still require further approvals?
On 30 Aug 2023, at 13:43, S E R A Y A @.***> wrote:

@takahser requested changes on this pull request.
@livetreetechhttps://github.com/livetreetech
However, I will discuss it further with the team and we will definitely consider incorporating a similar random role assignment mechanism into our framework. (...)
Okay, sounds good to me.
We look forward to your positive response and are available for further discussions, including a scheduled call if you deem it necessary.
We usually expect our grantees to have a clear vision of what they want to build, so you'll have to deliver the final concept before we can move on here. By final, I mean that it should be in a state we're confident will work. If necessary, you may amend changes to the proposal at a later stage, if necessary. Usually, it's not a problem to get the amendment approved if you can make a good case on you want to change it and the overall proposal still makes sense, also w.r.t. price.
—
Reply to this email directly, view it on GitHubhttps://github.com/w3f/Grants-Program/pull/1818#pullrequestreview-1602618415, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGXTVY3UZKN3DXHDXK6WEK3XX4YNJANCNFSM6AAAAAAZQR7XY4.
You are receiving this because you were mentioned.Message ID: @.***>

@takahser @semuelle
Thank you for your detailed feedback and for clarifying the approval process. Appreciate your openness to potentially approving our proposal upon the inclusion of a more robust incentivisation scheme. To address this, I added a specific deliverable to our grant proposal under "Milestone 2 — Collective Economy and Client Interface," which outlines the implementation of a random role assignment mechanism. This mechanism will be part of our example game logic and will be configurable using the Strategy pattern, thereby directly addressing your "game theory" concerns.
We believe that this addition not only strengthens our proposal but also aligns it more closely with the innovative approaches in the field, such as those presented in the VBFL paper. We are committed to delivering a high-quality, impactful project that will contribute significantly to the ecosystem. Again we understand that this on a milestone basis.
However, we must express our concern regarding the time taken for this evaluation process and the lack of transparency we've encountered. While we understand the need for thorough evaluation, the extended timeline and unclear status updates have been unlike any other grant application process we've experienced. Time is of the essence for us, and we believe for the community as well, to move forward with implementing these valuable contributions.
Given these updates and our proactive approach to addressing the committee's concerns, we kindly urge you to expedite the remaining steps of the evaluation process. We request that the proposal now be moved forward for review and final sign-off by the other evaluators. We are confident that we have met the criteria for approval and are eager to commence work on this promising project.
Thank you for your time and consideration. We look forward to a favorable resolution and to contributing to the community.

@takahser @semuelle
Thank you for your latest feedback. Clarification made to the "Milestone 2 — Collective Economy and Client Interface" section to directly address your concerns about role specialisation etc... We understand that the proposal is now in "ready for review" status, and assume that other committee members have been awaiting the closure of this thread to proceed with their evaluations. We kindly request that they now take the time to review our updated proposal. We have demonstrated our ability to adapt and address any questions or concerns, and we are eager to move forward. We are keen to proceed to the implementation phase. We look forward to a timely resolution and hope to receive the necessary approvals soon. Many thanks.

@takahser @semuelle - hope you had a nice weekend. I have updated the proposal simplifying the description, essentially, it will work the same way as the VBFL paper describes. Hopefully this finally enables us to close this conversation and move on.  Look forward to a timely resolution and hope to receive the necessary approvals soon.

I have read and hereby sign the Contributor License Agreement.

Thank you @takahser . I spoke with Dr. Ward about this, and we understand your concerns about the alignment of roles and mechanisms with the VBFL paper. However, it's important to clarify a few points:
Firstly, we've conducted extensive research, including the VBFL paper among other references, to craft a detailed proposal that we believe showcases our deep understanding of ML. We've made every effort to address your concerns, particularly regarding incentive mechanisms and integration with Substrate. Our expertise extends beyond the basic decentralisation validation outlined in the VBFL paper, and we hope you'll acknowledge the breadth and depth of our knowledge in ML (that you might be lacking or leading to confusion).
Our roles are designed to align with a standard ML workflow, aiming to create a framework that ML developers can readily use. Take the "Model Engineer" role as an example; it's a specialised role that can't be automated and requires an engineer with a deep understanding of ML to fine-tune parameters or introduce new algorithms to improve the model. To incentivise these roles, we're proposing a straightforward pool incentive mechanism. This mechanism will be determined by what the Model Creator deems best for the model's development and will be governed collectively on-chain. This flexible approach, we feel, syncs with how model development operates now and leverages a simple yet powerful on-chain governance and tokenisation extension.
Validation and Game Theory: We introduced roles like "machine learning worker," "model validator," and "blockchain miner" to address the  "game theory" concern which seemed to be your focus. In our example Strategy, like the VBFL paper, the roles can be dynamically assigned, to help prevent malicious behavior. While we appreciate the VBFL paper, we see it as just single Strategy out of many. These Strategies would be dependent on what the Model requires to be best improved e.g. extended with cross-validation techniques like k-fold and stratified sampling. You asked for specifics on an example of how to implement a trustless validation, we provided the VBFL paper and how it would work. Please accept our proposal aims to go beyond the VBFL paper. We're focusing on decentralisation over model development, which adds a new paradigm to ML model development.
Committee Review: We feel we have made considerable effort to address the concerns and are confident in our ability to deliver a valuable product. We've agreed to a milestone-based approach, which minimises risk and ensures that we deliver value. Given the relatively low risk (Level 2 grant) and high transformative potential, we suggest scheduling a call to address any remaining concerns that the other committee members may have. We're concerned that detailed written answers back/forth might be confused as there are some cross-over in terminologies e.g validation in ML is quite different from this back and forth. Anyhow, we hope to open this opportunity up to other committee members to review and together deliver the transformative potential of our proposal.
We look forward to moving this proposal to the next stage and are open to further discussion to clarify any points. Thank you for your time and consideration

Hi @livetreetech. Could you let us know your availabilities early next week, so I can schedule a call with members of the committee? Feel free to email me, including email addresses from your team to invite.

Would this be able to support custom hardware for machine learning like NVIDIA DGX systems or tensor cores?

Thanks @dsm-w3f  good input, may I suggest we have another call, if the following doesn't address your concerns

There is no comparison with other solutions and approaches for the same problem. I was able to find this solution for the same theme (https://feltlabs.ai/). Could you compare your proposal with what they are doing?

Feltlabs seems to be closely linked with the Ocean protocol based on the documentation. It uses Ocean’s reward system for incentives and seems to be specifically designed to fit within the Ocean protocol, utilizing algorithms like FedAvg. So, it’s not a broad framework like Tensorflow/DecentralML but more of a specific use case, built on top of Ocean’s reward protocol. On the other hand, our DecentralML is a flexible framework. It expands on Tensorflow FL and incorporates features like tokenisation and governance, all within the Polkadot ecosystem, allowing for broader applications.

Flexibility. Furthermore, there is literature under the terms Blockchain-based federated learning (BCFL) and Blockchain-empowered Federated Learning. In special, looks like there are many approaches to dealing with incentive models in BCFL. In special, these papers 1 - see section 3.d and 2 - Section 6 mention some of them, but there are more in the literature. How does your proposal fit in the current literature of incentive models for BCFL?As mentioned, there are many incentive models for BCFL. Would be possible for your proposal to be flexible and accommodate incentive models such as a framework? In this way, would fit better as a common good and the proof that the model works is the responsibility of each instance model. Using a already proof model to show that the framework works also would be nice to avoid skepticism about what you are proposing.

DecentralML is designed for flexibility, utilising the Strategy pattern we've previously discussed at length. This allows it to adapt to various approaches and reward models in Blockchain-based federated learning, such as VBFL. I agree it’s important for the framework to be adaptable and evolve with advances.

End-to-end (e2e) testing. How do you plan to show that the complete system works? What will be the case to show the federate learning and incentive model working? I think the testing part says unit tests and in the documentation somewhat manual, but when we evaluate a system we need to see all parts working together. Please add e2e testing guide as a deliverable of your proposal and explain what will be the case to show the system working.

I've included training using the MNIST dataset; it’s a suitable starting point for FL classification. I have updated the grant proposal deliverables: "The guide will have Testing and End-to-End (e2e) Testing... demonstrating federated learning tests using the MNIST dataset for classification."

I'm interested in the results of this, so I'm giving my approval.

Great! Thank you for the support @semuelle

Would this be able to support custom hardware for machine learning like NVIDIA DGX systems or tensor cores?

Absolutely! The Tensorflow library was chosen partly because of significant investments in hardware optimisation, specifically around GPUs and TPUs, through various distribution strategies. Happy to discuss this further on a call if more details are needed.

@livetreetech thank you for the answer and improvement. As far as the incentive strategy is flexible, it can be adjusted and the game theory for showing the incentive model works it is responsibility of the model creator. Having e2e tests enables us to verify the software. Please consider when implementing the test plan to include test cases to show how to change the incentive model using the strategy pattern. I'm considering that other parts of the incentive model will also use strategy pattern and will have test cases to show its flexibility. Correct me if I'm wrong. Considering that, I'm happy to go forward with it.

@dsm-w3f good point we will include tests to demo the change of incentives and the other parts of the incentive model within the strategy.  Thank you for the support.

@Noc2, many thanks! We appreciate your support!
Our discussion back and forth on the project has been lengthy but informative, and we are excited to bring various PMFs around what this project delivers i.e. tokenisation and governance over foundational brand specific models. We are currently looking at next month to fully release resource. The conversation has created moving timelines around the grant but we have been progressing and developing various use cases.  Thank you for the understanding!

@livetreetech PMFs as in "Product-Market Fits"?
Regarding the timeline, if you're experiencing delays it's usually not a problem, as long as you communicate with us that you're still working on it and if you - once you exceed the deadline - amend your proposal accordingly. 👍

Hope this finds you all well! Just to note, we have made a start on the project and added Mathias Ciliberto (https://www.linkedin.com/in/mciliberto/). We will be making commits and commenting here were appropriate during the implementation. Thanks again for the support.

@livetreetech thanks for keeping us in the loop, appreciated. Just for completeness of content, you amend your proposal and add him to the team? Should be easy and quick to approve from our side. 👍

Thanks @takahser I did commit the changes (application md attached) - I dont see why it's not reflected.  If possible, to save some time please can you commit it from your side (and send some instructions if you get a minute, Ill play around with it asap)?
decentral_ml.md

Hey, hope this finds you well - we have pushed over the delivery milestone 1 and are excited with how the implementation is going, we have good momentum: https://github.com/w3f/Grant-Milestone-Delivery/pull/1079 When you get a minute, can you please give us an indication on when you'll have time to review - we will would like to avoid any delays or pauses in development ? Thank you.

@livetreetech sorry, I didn't reply to your comment regarding the amendment earlier. If you update your fork and create a new pull request, it should work fine.
Regarding the milestone delivery, thanks for the submission. Unfortunately, this month some team members are out of office for various reasons, hence we're a big behind our backlog. But usually we're giving our teams initial feedback within 2 weeks after milestone submission. LMK if you have any further questions.

Thank you for your update @takahser I've just submitted a new pull request adding Dr. Ciliberto.
I wanted to bring our project's current situation to your attention, the delays you've outlined on your side put us in a tricky position. Our team, including highly paid, sought-after, PhD and professionals, is dedicatedly working on this two-month project. As you can understand, halting their work for an 2+ weeks in a two-month project is not feasible, both practically and financially. Each day of delay adds significant risk and cost to our project. I appreciate the usual process and timelines you follow, but considering our circumstances, I kindly ask if there is any way to expedite the review? We're confident in the quality of our work and you can see the code and concept is solid. We are more than willing to assist in any way to facilitate a quicker evaluation. If it helps, we're open to arranging a call to walk you through our code and address any concerns you might have?  Thank you for the understanding.

Hi @livetreetech thanks for your patience, as a majority of the team was out last week. Milestones can be submitted concurrently, so you don't have to wait for this one to be reviewed before moving onto the next one. Therefore you shouldn't have to delay development, if the next milestone is ready you can go ahead and submit it. I hope this helps!

Hey @livetreetech. I will try to review the milestone tomorrow. I'll let you know if I have any questions.


What does the level argument in the Rotate transformation correspond to?

In mmdectection/mmdet/datasets/pipelines/auto_augment.py
the Rotate class is used to apply a transformation to the image. What exactly does the level argument correspond to? In my case a high value of level quickly leads the model to a very low accuracy. Thank you very much.

Feel free to reopen the issue if there is any question.

Hi, @vblua  level represents the discrete rounding level of the augment operation, the larger the value, the stronger the augment.