Sogol Fathian

Tag: product management

  • Building an AI vs a regular Product

    I’m learning more about building products infused with AI. What does an AI product mean and how building it is different from building a regular product? You can differentiate an AI product vs a traditional product through the following criteria:

    AI product starts with existing technology

    Traditional product development cycle starts with understanding a specific customer or business problem. Once the problem is identified then Product, Design and Eng work together to build a solution that satisfies that problem. However contrary to traditional product an AI product starts with available technology and it’s Product Manager job to find use cases solving or improving customer or a business problem using that technology. I heard from a PM who worked at Google that for a long time they had voice-to-text technology but didn’t know what to do with it. They iterated quite a bit from trying search by voice in browser to other features until they developed Google Home, a smart speaker designed to respond to voice commands. Though voice-to-text is not much used in browser because of easy access to keyboard, voice control is now an integral tool in our everyday tasks to search on TV or interact with smart devices.

    AI is here to make the solution smarter

    While a traditional product provides solution to address pain points, AI use cases are usually applied on top of it to make that solution “smarter”. This mean looking at patterns and trends within the existing data to to either optimize the solution, provide recommendation and insights or make the solution customized at scale. I use advertising industry where I work to provide an example of each use case.

    I work at a digital advertising platform that helps brands and agencies run advertising campaigns across internet. In advertising ecosystem content providers (such as New York Times, Netfelix, Walmart) initiate a bid request for ad placeholders on their website, videos, podcasts etc while companies like mine responds to those bid requests to place a variety of ads in those inventories:

    • Optimization: Advertisers have a budget to spend for campaigns and along with it a set of objectives and key performance indicators, such as maximizing conversions, clicks, or visibility. They want to ensure they maximize return on marketing dollars while achieving these goals. It’s the system responsibility to determine the most effective bid amount to maximize the chance of wining without overpaying. The system analyzes data such as historical performance data and user behaviour among others data types and uses machine learning algorithms to predict bid outcomes and make real-time adjustments based on this data and campaign goals.
    • Recommendation: One of the main goal of advertising is to reach the right people looking for the solutions/services advertisers offer. Advertisers have specific criteria on who to target based on geographic location, demographics, user’s action etc. To expand their customer reach, advertisers look for recommendations on other groups of people to target. The system use the characteristics, behaviours, and interests with an existing audience and use them to identify and recommend new users who closely resemble the seed audience based on these characteristics.
    • Personalization: Personalization has become increasingly important in advertising because personalized ads increases consumer engagement, brand loyalty, and overall marketing effectiveness. Machine learning solutions can be used to create various versions of a creative and messages based on user’s data and past behaviours in mass scales.

    AI solutions have existed for a long time without being specifically called AI, while generative AI solutions exploded more recently with introduction of chat GPT and other large language models.

    Many AI solutions have minimal User Interface

    While traditional products have strong element of UI where the user gets to interact with it to derive the value, AI solutions usually don’t offer much of a UI. While massive amount of data and models goes into building and generating optimization, recommendation or personalization services they’re usually behind the scenes and is transparent to the user. AI recommendation Generative AI interaction is conversational text based and is mostly done through the prompts, voice though the field is quickly evolving and we may witness more interaction.

    Having above categories have helped me to quickly identify if I’m dealing with a traditional product or if the product is AI based. I hope it helps you too.

  • Writing Technical Stories

    Recently I gave a talk about writing technical stories and ways I have found to slice user stories thinner at Product Camp Toronto. I thought it’d be good to share some ideas from that talk here as well.

    I work as product manager for epost which is one of the biggest online document delivery and management systems within Canada. epost allows users to view, pay and manage bills online and roughly one in ten Canadian use it to view their pay stubs, utility bills and tax statements among others. Right now I am in charge of leading a scrum team to redesign epost to improve web and mobile user experience. I am also responsible to revamp epost API service calls and to integrate them with Canada Post, banks and mobile apps.

    My team is one of the first teams working Agile within Canada Post, so we are all new to the concept of finishing user stories within a two week sprint. And after the first few sprints I realized that we were constantly behind schedule. Our flow for getting a story done looked something like the following:

    user story flow

    Creating API call and gateway call would take half sprint and front-end will work on the last leg of sprint to get it done. So when the story was ready for QA we were out of time and stories kept piling up to the next sprint!

    I also realized during our sprint planning and release planning a bunch of tasks get discovered and brought up that are not identified earlier. By the time these tasks are done, we are already late with the sprint. It was clear to me that I needed to write smaller stories but how?

    With a help of a my great Scrum Master Peter Moreira and User Story Mapping (I review the book in a future post) I came of with the following strategies.

    Write Technical Stories (when necessary)

    I decided since any given back-end service call will be consumed by multiple end-points it makes sense to slice a story further into an API-specific one and a front-end/UI one. API story is still valuable and independent (sort of) but it can be estimated and worked on by itself. This way I could prioritize the API story ahead of front-end story to provide much needed time for UI and QA to be done.

    This was great however I didn’t know how to write user stories about API, web services and other technical stories that are not necessarily user facing. More importantly I was struggling to fit the story into “As a user I want to — so that I can —” template.

    “As a front-end developer I need to supply mail list data so that it shows as a list view of mails” sure sounds weird!

    I came across this excellent article that confirmed my belief not to fit the story into a template when it doesn’t fit. So for technical stories I clearly write Who the end-points consumers of the API call will be, What is expected acceptance criteria of each of the end-points are and what it the expected output of this API is and finally Why this API call is important to develop. I talk to developers to understand what does API generate, how it can be validated and make sure there is alignment on acceptance criteria prior to grooming session. From my point the story is usually done once I can see API call and the data generated in Swagger and all the security criteria is met.

    So a user story that looked like this before:

    —As a Chief Household Officer, I want to filter mail by service provider name so that I more easily find my mail

    the API story will look like this:

    —We need an API call used by Canada Post, native app and our bank partners to filter mail by a service provider so only mails by that provider shows up

    Acceptance Criteria

    • Verify that the request get thru the service layers and receive a reply within 2 seconds
    • Verify that the request has necessary Oauth permission to be exposed externally
    • Verify that the request is generated in both XML and JSON formats
    • Verify that records are returned from oldest to newest

    —Separate Success and Error Stories

    Another trick I’ve found effective to write smaller stories is to focus one user story only on the happy-path, while writing other stories for when things went wrong, edge cases. For example for file creation user story write one story specifically about creating a file without any problem but write several more stories to cover the file name maximum character, acceptable  characters etc.

    I would love to know if you have other ways to slice a story further. Please share them with me in the comments below!

  • How agile is your team?

    Output vs Outcome

    I work at a big corporation that is transitioning to Agile. I know this may come as a shock to my readers but yes it’s true! Although many small companies and start-ups have embraced Agile and Lean ways of working for many years now, big corporations specially corporation not focused on technology are just testing the water when it comes to making this transition. It is a big change for many people who’ve worked the same way for many years.

    Given that my agile team is new and green I was thinking of ways to measure my team and be able to track the progress over each sprint. Here is what I’ve learned:

    The main goal of an agile team must be to minimize output while to maximize outcome and impact

    but what is the difference between output and outcome and how do you measure each?

    Output vs Outcome

    I’m reading the excellent User Story Mapping by Jeff Patton (a review soon to come) and here is how he describes the difference between Output and Outcome:

    “Everything between the idea and the delivery is called output. It’s what we build, but while it’s necessary the output isn’t the real point; it’s not the output that we really wanted. It’s what comes after as a result of that. It’s called outcome. We want to measure what people actually do differently to reach their goals as a consequence of what we’ve built.”

    Take a look at the image above for a moment and let it sink in. It’s crystal clear now isn’t it?

    Measuring Team’s Output

    At this post I’m only concentration on measuring Outputs like features, enhancements, requirements, specification within an Agile team. Measuring Outcomes is really about understanding bigger picture and product/market fit and I like to discuss it completely in a separate topic.

    So here are what I learned:

    For outputs the most common factor to measure is Velocity. Velocity is commonly referred to number of story points completed within each sprint; (and assuming this metric is not abused) it’s valuable as a predictive metric as it starts to normalize to show how much the team can produce in any given Sprint. The expectation is that over time an increased team velocity means shipping more features faster.

    However in my company we are still limited to predefined timelines for enterprise releases (I know this is an almost non-issue for startups) so for my team’s Velocity doesn’t necessarily translate to stories completed and ready to ship. At this point all we can do it to bundle stories completed and mark them as ready to go per release.

    Another metric that I found useful is the number of bugs generated during each sprint. Again over time we want to decrease this. My team is new to AngularJS and one thing that happened is that refactoring the code from one sprint to next broke a lot of things.

    Another interesting point learned from Quora is qualitative feedback we get during Retrospective meeting and use those improve upon. Pick something to improve and track that for a few iterations to make sure things are getting better. Once things have improved stop tracking, and move on to the next pain point. Again in my team’s case we’re trying to see what are the ways we can get rid of mini-waterfall cycle we seem to get trapped on every time. So far we’re concentrating on two issues: 1) increase expertise in Angular by having Angular hours with other developers. 2) interchange roles and responsibilities of back-end API and gate-way/middle-ware developers.

  • Why do you need to be a hands-on Product Manager?

    Product Management is a hybrid role and depending on where you work, your list of responsibilities vary. You need to have different skill-set to be able to build a product that is valuable, usable and feasible. Also, product manager responsibilities comes from working with multiple teams and within the frame-work of product development. So it’s hard to showcase product manager abilities in a stand alone context. What I mean by that is, a software developer has working code and a designer has portfolio to show for but what is a concrete output of a Product Manager?

    This is why it’s so important to be a hands-on in multiple aspects of the job, not only to demonstrate your skills in a practical way but able to remove roadblock you encounter effectively and efficiently.

    If you are working in Information Technology like I do, your day-to-day responsibilities change depending on which phase of software development life cycle you’re at. In early phase of software development life cycle you need to work effectively with multiple stakeholders but in order to do that you need to be 1) knowledgeable about product/market fit 2) opinionated about what needs to be built 3) be able to justify your opinion.

    If you are working with developers you need to speak their language and be able to understand what they’re saying and what are the implications of their proposal’s for the product will be. Are you able to understand and verify what is said? The same is true when you’re working with designers. When I first started, I worked with a designer who would get on implementing her ideas and I always justified that approach by thinking she’s the expert in her field and I should trust that judgment. I didn’t know about principles of user experience design nor did I about the importance of building customer persona before jumping right in to designing stuff. Now at least I can have an effective  discussion with UI/UX designer and ask questions on why the interface needs to be the way s/he emphasizes to be.

    When time is short and you need to get a product released in a tight deadline, there are so many little things to do. It’s so much more efficient to take on these tasks and free up experts (designers, developers) time to do the heavy lifting. Can you jump in as the copy-writer to change that campaign copy for the last time before it goes out, do you have some Photoshop skills tweak the layout, and write the code for the script to get the data you really want but Business Intelligence team doesn’t have time to do it now?

    All this hands-on experience becomes invaluable if you want to build your own business.  Do you have what it takes to get to that first MVP?